<meta lang="en" charset="utf-8"> 

**vk_mini_path_tracer: Extra Chapters**

Extensions and improvements for the path tracer described in [the main tutorial](index.html).

<small>
By Neil Bickford
</small>

<script type = "preformatted">

This page contains the *extra chapters*, showing how one can extend and improve the mini path tracer described in [the main tutorial](index.html)! We'll show how to extend the path tracer with multiple instances and materials, improve quality using nicer antialiasing, improve performance using images and ray tracing pipelines, and analyze app behavior and performance using Nsight Systems.

More specifically, you can use this page to learn how to:

- Improve antialiasing with a few lines of code using a **Gaussian filter** instead of a box filter;
- Visualize how the application works and **analyze performance** using Nsight Systems;
- Use **compaction** to make BLASes use less memory;
- **Share structures and constants** between C++ and GLSL by including polyglot files;
- Use **push constants** to change resolution without recompiling GLSL code;
- **Render thousands of samples** using multiple command buffers;
- Use **Vulkan images**;
- Add **names to objects** to help with debugging;
- Add **multiple instances** to render large amounts of geometry;
- Give instances **different materials**;
- Use **ray tracing pipelines** to accelerate multi-material scenes.

When optimizing an application, it's important to consider all levels of optimizations, including API-level optimizations -- things that help system components like the driver, GPU, and CPU complete work faster -- and higher-level optimizations, like [denoising](https://github.com/nvpro-samples/vk_denoise), [DLSS](https://developer.nvidia.com/dlss), path guiding, irradiance caching, and [better sampling techniques](https://eheitzresearch.wordpress.com/772-2/).
Since this is a Vulkan tutorial, this page will focus mainly on API-level tools and improvements, but there's a lot of interesting techniques for quality and speed at all levels of the stack. Regardless of the level one's looking at, the tool described in Chapter [Measuring Performance] can help one optimize performance and better learn how an application works.

In contrast to the main tutorial, where each chapter built on the previous one, the chapters in these extras can be approached sequentially or on their own (in which case you'll probably want to use the code checkpoints referenced at the start of each chapter). They also assume some familiarity with Vulkan, and may be a bit more complex than the main tutorial.

With that in mind, let's get started!

!!! Tip Link to Code and Chapter Checkpoints on GitHub
    This tutorial's text and code (including chapter checkpoints -- see *Debugging Help* in the [main tutorial](index.html))
    are available here: https://github.com/nvpro-samples/vk_mini_path_tracer

# Gaussian Filter Antialiasing

In the [*Antialiasing and Pseudorandom Number Generation*](index.html##antialiasingandpseudorandomnumbergeneration) chapter of the main tutorial, we implemented what is known as a [**box filter**](https://en.wikipedia.org/wiki/Window_function#Rectangular_window) for antialiasing. Informally, every point in a box (of side length 1 pixel) around the pixel's center has an equal chance of being chosen. Here's a plot of the distribution of 20000 randomly chosen antialiasing samples around a pixel center:

![The screen-space origins of 20000 samples around a pixel center chosen to follow a box filter. Axes are marked in units of pixels.](images/e1-boxFilter.png)

Although a box filter is a major improvement over no antialiasing at all, Gaussian filters -- which place more samples near the pixel center according to a [Gaussian distribution](https://en.wikipedia.org/wiki/Normal_distribution) -- are usually better for antialiasing, reducing [Moiré artifacts](https://en.wikipedia.org/wiki/Moir%C3%A9_pattern) as well as providing higher theoretical quality. Informally, this is because a Gaussian filter is generally a better low-pass filter than a box filter (see the *What is the Perfect Antialiasing Filter?* sidebar in [the main tutorial](index.html#antialiasingandpseudorandomnumbergeneration/antialiasing) for more information).

Here's a plot of the distribution of 20000 randomly chosen antialiasing samples using this chapter's Gaussian filter:

![The screen-space origins of 20000 samples around a pixel center chosen to follow a Gaussian filter. Axes are marked in units of pixels.](images/e1-gaussianFilter.png)

<!-- These images were generated using the following p5.js code:
lineWidth = 0.01/2;
ptRadius = 0.02;
tickSize = 0.1;
fontSize = 0.15;

function applyCS() {
  translate(width/2, height/2); // center is at the origin of the screen
  scale(width/4.5); // 4.5 units = 1 screen width
}

function setup() {
  createCanvas(800, 800);
  background(255);
  
  resetMatrix();
  applyCS();
  strokeWeight(lineWidth);
  textSize(fontSize);
  textAlign(CENTER, CENTER);
  line(-3, 0, 3, 0);
  line(0, -3, 0, 3);
  textFont('Trebuchet MS');
  
  for(d = -2; d <= 2; d++){
    if(d != 0){
      stroke(0);
      // x-axis
      line(d, -tickSize, d, tickSize);
      // y-axis
      line(-tickSize, d, tickSize, d);
      
      // text
      noStroke();
      fill(0);
      text(d.toString(), d, -2 * tickSize);
      text(d.toString(), 2 * tickSize, -d);
    }
  }
}

numPtsDrawn = 0;
drawnPerFrame = 100;

function drawPt(x, y){
  ellipse(x, -y, ptRadius, ptRadius);
}

function draw() {
  resetMatrix();
  applyCS();
  noStroke();
  fill(0, 0, 0, 48);
  if(numPtsDrawn < 20000){
    for(i = 0; i < drawnPerFrame; i++){
      u = random();
      v = random();

      // Box
      /*
      x = u - 0.5;
      y = v - 0.5;
      */
          
      // Box-Muller with sd = 0.375
      u = max(1e-38, u);
      r = 0.375 * sqrt(-2.0 * log(u));
      theta = 2 * PI * v;
      x = r * cos(theta);
      y = r * sin(theta);

      drawPt(x, y);
      numPtsDrawn += 1;
    }
  }
}
-->

It turns out that switching to a Gaussian filter instead of a box filter uses only a few lines of GLSL, using the [Box-Muller transform](https://en.wikipedia.org/wiki/Box%E2%80%93Muller_transform).

!!! Tip Troubleshooting
    If you run into any issues during this chapter, you can compare your code
    against the reference code at the end of this chapter provided in the `checkpoints/e1_gaussian` folder. For instance,
    you can use a diff tool such as [Meld](https://meldmerge.org/) or [Diffchecker](https://www.diffchecker.com/) to automatically find any differences between
    your code and the reference code.

Here's how to implement a Gaussian filter. Starting from the code in `vk_mini_path_tracer`, add the following code after the definition of `stepAndOutputRNGFloat` in `raytrace.comp.glsl`:

~~~ glsl
const float k_pi = 3.14159265;

// Uses the Box-Muller transform to return a normally distributed (centered
// at 0, standard deviation 1) 2D point.
vec2 randomGaussian(inout uint rngState)
{
  // Almost uniform in (0,1] - make sure the value is never 0:
  const float u1    = max(1e-38, stepAndOutputRNGFloat(rngState));
  const float u2    = stepAndOutputRNGFloat(rngState);  // In [0, 1]
  const float r     = sqrt(-2.0 * log(u1));
  const float theta = 2 * k_pi * u2;  // Random in [0, 2pi]
  return r * vec2(cos(theta), sin(theta));
}
~~~

This defines the value of $\pi$ (which we'll reuse later on), and then defines a function that generates two uniformly distributed random numbers `u1` and `u2` between 0 and 1, then uses the [Box-Muller transform](https://en.wikipedia.org/wiki/Box%E2%80%93Muller_transform) to make these follow a normal distribution with standard deviation 1 centered at (0,0).

Then replace this line inside `void main()`:

~~~ glsl
    const vec2 randomPixelCenter = vec2(pixel) + vec2(stepAndOutputRNGFloat(rngState), stepAndOutputRNGFloat(rngState));
~~~

with the following:

~~~ glsl
    // Use a Gaussian with standard deviation 0.375 centered at the center of
    // the pixel:
    const vec2 randomPixelCenter = vec2(pixel) + vec2(0.5) + 0.375 * randomGaussian(rngState);
~~~

This code scales the random `vec2` to have a standard deviation of 0.375 (which is approximately what the Blender Cycles renderer uses for its Gaussian filter), then positions it over the center of the pixel, which is at `(pixel.x + 0.5, pixel.y + 0.5)` -- midway between `(pixel.x, pixel.y)` and `(pixel.x + 1, pixel.y + 1)`. (This is because `pixel` is the index of the pixel, giving the coordinate of its top-left corner, while the center of the pixel is at a half-integer. The previous line's random `vec2`s were centered at `(0.5, 0.5)`, while `randomGaussian`'s `vec2`s are centered at `(0.0, 0.0)`.)

Finally, we use $2\pi$ in the diffuse reflection code, so let's replace it with `2.0 * k_pi`, by replacing

~~~ glsl
        const float theta = 6.2831853 * stepAndOutputRNGFloat(rngState);  // Random in [0, 2pi]
        const float u     = 2.0 * stepAndOutputRNGFloat(rngState) - 1.0;  // Random in [-1, 1]
~~~

with

~~~ glsl highlight
        const float theta = 2.0 * k_pi * stepAndOutputRNGFloat(rngState);  // Random in [0, 2pi]
~~~ glsl
        const float u     = 2.0 * stepAndOutputRNGFloat(rngState) - 1.0;   // Random in [-1, 1]
~~~

You should now be able to build and run the code, generating a new antialiased image. The differences will probably be relatively subtle for this scene -- look closely at edges of meshes and at the horizon -- but this Gaussian filter will avoid artifacts for scenes with finer detail later.

![The path traced image from the main tutorial.](images/12-vk_mini_path_tracer.png)

![This chapter's image with Gaussian filter antialiasing.](images/e1-gaussianPT.png)

![Difference between the last two images in sRGB space, times 2.](images/e1-gaussianDiff.png)

![Cropped and scaled look at the horizon in these two images; the Gaussian filter's larger sampling range can be seen on the left.](images/e1-gaussianScale.png)

To see the effect more clearly, we can turn this into more of a Gaussian blur than an antialiasing filter by scaling `randomGaussian` up by a factor of 100:

~~~ glsl
    // Use a Gaussian with standard deviation 37.5 centered at the center of
    // the pixel:
    const vec2 randomPixelCenter = vec2(pixel) + vec2(0.5) + 37.5 * randomGaussian(rngState);
~~~

![The image with a Monte Carlo Gaussian blur of standard deviation 37.5.](images/e1-gaussianBlur.png)

Make sure to revert this change by changing `37.5` back to `0.375` before continuing.



# Measuring Performance

Measuring performance is often one of the most important steps in optimizing an application. Without measuring, it's hard to tell where the system spends its time working, which can be surprising -- a ray tracing application might turn out to spend most of its time waiting for the CPU to finish instead of processing data on the GPU, or might spend more time transferring data than ray tracing, for instance.

In this chapter, we'll visualize where the path tracer spends its time using [NVIDIA Nsight Systems](https://developer.nvidia.com/nsight-systems), a performance analysis tool that visualizes both CPU and GPU work at the same time. It's also possible to instrument a Vulkan application using [timestamp queries](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/vkCmdWriteTimestamp.html) in [query pools](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VkQueryPool.html), but a performance analysis tool is often easier to get started with, and can provide hard-to-find info.

When working with a new app, we usually recommend starting analysis with Nsight Systems to get a view of how the application uses the entire system. (It often turns out that an application could run faster if it moved some of its work on the CPU to the GPU, could run faster if it used the CPU and GPU in parallel instead of sequentially, or is bottlenecked by system calls.) Depending on where the app's performance is bottlenecked, we might then suggest using Nsight Graphics (for graphics APIs such as DirectX and Vulkan), Nsight Compute (for CUDA), or a CPU performance analysis tool.

![](images/e2-triangle.png)

Because of the way this application works, we won't get as much information from Nsight Systems as we would for a more typical app (with a swapchain, for instance) -- but we'll be able to infer the information we'll need for this tutorial. This chapter will be updated as the capabilities of Nsight Systems and Nsight Graphics change.

!!! Tip
    This chapter has no new code -- it's entirely about analyzing the code we have using Nsight Systems! You can use the code checkpoint in the `checkpoints/e1_gaussian` folder.

## Preparing for Analysis

First, make sure to make a release-mode build of your project. Nsight Systems can work with debug-mode applications -- but release mode will usually make the application run faster.

Make sure to download and install the latest version of Nsight Systems from https://developer.nvidia.com/nsight-systems. The images and data below were all captured on a system with an RTX 3090 running Windows 20H2 and Nsight Systems 2020.5.1.

At launch, Nsight Systems will ask whether you want to view GPU or CPU rows on top. I've chosen the "GPU Rows on Top (new)" mode.

![A screenshot of the Nsight Systems "Options Preset" window.](images/e2-optionsPreset.png)

## Profiling an Application

In the main window, one should see the Project Explorer with a list of projects (like folders that can contain multiple performance captures) on the left, and a tab on the right. By right-clicking on the project and selecting "Rename...", one can give the project a useful name; I've named it "Vk_mini_path_tracer Analysis".

![The main Nsight Systems window.](images/e2-mainWindow.png)

Next, tell Nsight Systems to launch the application and collect performance data on the current machine by clicking the "Select" link in the center, or selecting the current machine from the "Select target for profiling..." dropdown. You may need to accept the User Account Control dialog (on Windows) to initialize the Nsight Systems tools app.

The main window should now look like this:

![The main window displaying trace options.](images/e2-traceOptions.png)

Here, we'll set the analysis options, choosing the application to profile, how to collect CPU-side performance data, and which APIs to trace (since Nsight Systems can show how an app uses multiple APIs at once).

For this application, set "Command line with arguments" to the path to your release-mode application. Check "Collect Vulkan trace", and uncheck "Collect WDDM trace" if you see it (since we won't look at Windows Display Driver Model events in this tutorial). I've also expanded the "Collect Vulkan trace" section so that you can see what options are available.

![The main window with all trace options set.](images/e2-setSettings.png)

Then press Start! You should see the console window for the path tracer pop up; once it finishes, Nsight Systems will collect and analyze performance data to produce a report.

![Nsight Systems processing samples.](images/e2-analysis.png)


## Examining the Report

!!! WARNING
    It's important to note that this is a toy path tracer, so it may have some unusual performance characteristics compared to larger applications! In particular, it doesn't use a swapchain, and we'll see some unusual timing the first time a few different types of resources are used. Conclusions from this report might not hold for other projects, which should be analyzed independently.

By default, Nsight Systems will show the Timeline View and the Events View, which will look like this:

![The Timeline View and Events View.](images/e2-timelineEvents.png)

The Timeline View charts system usage and traced API calls over time.

We can also click on the combo box at the top left of the report tab to switch between the Timeline View and the Diagnostics Summary (which shows messages from performance measurement), the Analysis Summary (which shows information about the system and collected samples), the Symbol Resolution Logs, and Files.

![The Diagnostics Summary shows 92 collected Vulkan events.](images/e2-diagnostics.png)

![The Files tab can show the captured stderr and stdout logs.](images/e2-logs.png)

Here, we'll look at the Timeline View to get an idea of where the application is spending its time. This view has dropdowns for overall CPU utilization (which can be expanded to see utilization by core), GPU utilization (which can be expanded to see VSYNC events), and then rows for each of the processes and their threads. In this case, we'll look at the first thread in the `vk_mini_path_tracer_e1_gaussian.exe` process.

![A closer look at the `vk_mini_path_tracer_e1_gaussian.exe` process.](images/e2-process.png)

!!! Tip
    Note that this only shows CPU behavior for the process, not GPU behavior (due to how this application works)! However, we'll be able to infer the performance characteristics we need because of the way the CPU waits on the GPU in this application.

Looking at the "Vulkan API" row for the first thread, we can immediately see that after about 381 ms of startup, most of the application time is spent inside `nvvk::Context`'s call to `vkCreateDevice` to create the `VkDevice`.

To make these measurements more precise, we can right-click on the "Vulkan API" row, then click "Show in Events View".

![](images/e2-showEvents.png)

This will list all 66 Vulkan API events in the Events View below. We can also see when each event started, the duration of the event, and can often get a stack trace as well. In this case, we can see that the call to `vkCreateDevice` took 989.632 ms -- nearly a second.

![The Events View with the call to `vkCreateDevice` selected.](images/e2-vkCreateDevice.png)

We can see that this thread is performing a good amount of work during `vkCreateDevice`, and we can get an idea of where it's spending this time by clicking on the thread's row, then hovering over one of the golden sampling points.

![Hovering over one of the sampling points in the first half of the call to `vkCreateDevice`.](images/e2-samplingPoint.png)

We can also show the information from these sampling points in the Events View by right-clicking on the thread and selecting "Show in Events View".

!!! Tip
    One way to speed up this `vkCreateDevice` call is to reduce the number of
    queues the `VkDevice` can access. `Nvvk::Context` allocates the maximum
    possible, but most applications will require fewer queues. One way to do
    this is to change the following line in `nvpro_core/context_vk.cpp`
    in the nvpro-samples library:
    
    ~~~ cpp
    queueInfo.queueCount       = m_physicalInfo.queueProperties[i].queueCount;
    ~~~
    
    to (for instance):
    
    ~~~ cpp
    queueInfo.queueCount       = 1;
    ~~~

Let's get a better idea of what's happening in most of the Vulkan code we wrote by zooming in on the section between `vkCreateDevice` and `vkDestroyDevice`! To do this, select the range to zoom into by clicking and dragging over the Timeline View:

![Selecting a range (light green) by clicking and dragging.](images/e2-zoom1.png)

Then right-click the highlighted range and select "Filter and Zoom in".

![Filtering or zooming into a region.](images/e2-zoom2.png)

This will both zoom into the region, and filter events in the Events View to only those events within the filtered region.

![The Timeline View after zooming and filtering.](images/e2-zoom3.png)

We can see that there are a few Vulkan calls at the beginning of this region (which we'll look at shortly), followed by a long period of CPU work from about 1.430s (seconds) to 1.455s. To figure out what this work could be, we look at events 61 and 62 in the Events View:

![Examining two events in the Events View to determine where CPU time is being spent.](images/e2-eventsHDR.png)

We can see that this work is occurring between `vkMapMemory`, which ends at 1.43033s, and `vkUnmapMemory`, which starts at 1.45449s, and occurs just before we destroy the acceleration structures. This corresponds to the call to `stbi_write_hdr`, which means that (at the moment) `stbi_write_hdr` is taking up most of the time between creating and destroying the device!

~~~ cpp
  void* data = allocator.map(buffer);
~~~ cpp highlight
  stbi_write_hdr("out.hdr", render_width, render_height, 3, reinterpret_cast<float*>(data));
~~~ cpp
  allocator.unmap(buffer);
~~~

We could confirm this and investigate this further using a CPU profiler.

For now, let's look at this section of time, containing most of the Vulkan API calls:

![Zooming into a smaller range of time containing most of the Vulkan API calls.](images/e2-zoomRange.png)

![The time range from 1.369s to 1.431s after zooming and filtering.](images/e2-apiCalls.png)

Here, we can see the effects of most of the Vulkan code we've written. You'll also notice that the "Vulkan API Command Creation" row above shows the time from beginning to ending each of this application's four command buffers (and that the first command buffer takes longer to create than the other three). Here, I've zoomed into the region around the fourth command buffer.

Recording this small command buffer takes 33.116 us (microseconds, millionths of a second). We can also see the calls to:
- `vkAllocateCommandBuffers`(from `AllocateAndBeginOneTimeCommandBuffer`, 672 ns [nanoseconds, billionths of a second]);
- `vkBeginCommandBuffer` (from `AllocateAndBeginOneTimeCommandBuffer`, 4.284 us);
- `vkCmdBindPipeline` (9.333 us);
- `vkCmdBindDescriptorSets` (1.551 us); and
- `vkEndCommandBuffer` (2.689 us; all timings from the Events View for the Vulkan API row).

![A view of the timeline around the fourth command buffer.](images/e2-cb4.png)

Examining these command buffers more closely, we can see that

1. The first command buffer is `uploadCmdBuffer` in `main()`; we can see the calls to `vkAllocateMemory` from the two `allocator.createBuffer` calls. (This command buffer probably takes longer to begin than the others because it's the first command buffer in the application.)
2. The second command buffer comes from `raytracingBuilder.buildBlas`. As of this writing, Nsight Systems doesn't show the acceleration structure building calls for the official KHR ray tracing extensions here quite yet.
3. The third command buffer comes from `raytracingBuilder.buildTlas`.
4. The fourth command buffer is `cmdBuffer` in `main()`, which includes the `vkCmdDispatch` call.

!!! Tip Making Analysis Easier with NVTX Markers
    We could include NVTX (NVIDIA Tools Extension SDK) ranges in our application to label ranges of work directly from the application - so for instance, we could add a range saying that the second and third command buffers are for building acceleration structures, and another range saying that the fourth command buffer is for path tracing! This especially helps with larger code bases.

Zooming out, we can see two larger ranges of time when the main thread is blocked (has an item on the "Blocked State" row) and is waiting for something to complete: for 9.5ms starting at 1.393s, and for a total of 23.745ms starting at 1.406s. From the Vulkan API row, we can see that in both of these cases, the thread is blocked from continuing by a `vkQueueWaitIdle` (i.e. the thread is waiting for the GPU's GCT queue to finish its work) -- and these correspond to building the BLAS, and the ray tracing compute shader (performing 64 samples per pixel with a ray depth of 32). Normally, these would be excellent times for the CPU to perform other work.

![The time range from 1.369s to 1.431s ms.](images/e2-apiCalls.png)

We'll return to this topic and make more performance analyses in future chapters as we add new features.



# Compaction

After being built, a BLAS can be **compacted**, making it use much less memory, with no performance downside. Compaction is fast, and we recommend compacting all BLASes that are static (whose geometry doesn't change each frame) in our [RTX Best Practices](https://developer.nvidia.com/blog/rtx-best-practices/).

!!! Tip
    You can find the reference code at the end of this chapter in the `checkpoints/e3_compaction` directory.
    If you run into any problems with the code in this chapter, you can diff your
    files against the reference files to see if there are any differences.

To build BLASes with compaction using the `nvvk::RaytracingBuilderKHR` helper, all we need to do is add `VK_BUILD_ACCELERATION_STRUCTURE_ALLOW_COMPACTION_BIT_KHR` to `nvvk::RaytracingBuilderKHR::buildBlas`' flags:

~~~ cpp
  raytracingBuilder.buildBlas(blases, VK_BUILD_ACCELERATION_STRUCTURE_PREFER_FAST_TRACE_BIT_KHR
~~~ cpp highlight
                                          | VK_BUILD_ACCELERATION_STRUCTURE_ALLOW_COMPACTION_BIT_KHR);
~~~

You should now be able to run the application and see it print a line like this in the console:

~~~ output
 RT BLAS: reducing from: 6912 to: 2048 = 4864 (70.37% smaller)
~~~

This tells us that compaction reduced the size of the BLAS from 6912 bytes to 2048 bytes, saving 4864 bytes and making it 70.37% smaller.

!!! Tip How Does This Work?
    Internally, compacting a BLAS involves six steps:
    
    1. Build the original BLAS with the `VK_BUILD_ACCELERATION_STRUCTURE_ALLOW_COMPACTION_BIT_KHR` flag.
    2. Use `vkCmdWriteAccelerationStructuresPropertiesKHR` to query the size of the compacted BLAS, and store the result in a [query pool](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VkQueryPool.html).
    3. Wait for the command buffer to finish and get the sizes from the query pool.
    4. Allocate space for a new BLAS with that number of bytes.
    5. Call `vkCmdCopyAccelerationStructureKHR` to copy the old BLAS into the new BLAS, setting `VkCopyAccelerationStructureInfoKHR::mode` to `VK_COPY_ACCELERATION_STRUCTURE_MODE_COMPACT_KHR` to compact along the way.
    6. Destroy the uncompacted BLAS.
    
    Here are some details of the implementation in `raytraceKHR_vk.hpp`. To create compacted BLASes,
    
    ~~~ cpp
    nvvk::RaytracingBuilderKHR::buildBlas
    ~~~
    
    first passes its flags to the `VkAccelerationStructureBuildGeometryInfoKHR` flags:
    
    ~~~ cpp
    buildInfos[idx].flags                    = flags;
    ~~~
    
    It then detects if it should output the uncompacted or the compacted BLASes, and creates a query pool:
    
    ~~~ cpp
    // Is compaction requested?
    bool doCompaction = (flags & VK_BUILD_ACCELERATION_STRUCTURE_ALLOW_COMPACTION_BIT_KHR)
                        == VK_BUILD_ACCELERATION_STRUCTURE_ALLOW_COMPACTION_BIT_KHR;

    // Allocate a query pool for storing the needed size for every BLAS compaction.
    VkQueryPoolCreateInfo qpci{VK_STRUCTURE_TYPE_QUERY_POOL_CREATE_INFO};
    qpci.queryCount = nbBlas;
    qpci.queryType  = VK_QUERY_TYPE_ACCELERATION_STRUCTURE_COMPACTED_SIZE_KHR;
    VkQueryPool queryPool;
    vkCreateQueryPool(m_device, &qpci, nullptr, &queryPool);
    ~~~
    
    Then, if it needs to do compaction, it does the following. First, it calls the Vulkan function to write the compacted size into the query pool:
    
    ~~~ cpp
    vkCmdWriteAccelerationStructuresPropertiesKHR(cmdBuf, 1, &blas.as.accel,
                                                  VK_QUERY_TYPE_ACCELERATION_STRUCTURE_COMPACTED_SIZE_KHR, queryPool, idx);
    ~~~
    
    To retrieve the size results from the query pool, it uses `vkGetQueryPoolResults`:
    
    ~~~ cpp
    // Get the size result back
    std::vector<VkDeviceSize> compactSizes(nbBlas);
    vkGetQueryPoolResults(m_device, queryPool, 0, (uint32_t)compactSizes.size(), compactSizes.size() * sizeof(VkDeviceSize),
                          compactSizes.data(), sizeof(VkDeviceSize), VK_QUERY_RESULT_WAIT_BIT);
    ~~~
    
    Then for each BLAS, it allocates space for a compacted acceleration structure:
    
    ~~~ cpp
    // Creating a compact version of the AS
    VkAccelerationStructureCreateInfoKHR asCreateInfo{VK_STRUCTURE_TYPE_ACCELERATION_STRUCTURE_CREATE_INFO_KHR};
    asCreateInfo.size = compactSizes[idx];
    asCreateInfo.type = VK_ACCELERATION_STRUCTURE_TYPE_BOTTOM_LEVEL_KHR;
    auto as           = m_alloc->createAcceleration(asCreateInfo);
    ~~~
    
    Then it calls `vkCmdCopyAccelerationStructureKHR` to create a compacted copy:
    
    ~~~ cpp
    // Copy the original BLAS to a compact version
    VkCopyAccelerationStructureInfoKHR copyInfo{VK_STRUCTURE_TYPE_COPY_ACCELERATION_STRUCTURE_INFO_KHR};
    copyInfo.src  = m_blas[idx].as.accel;
    copyInfo.dst  = as.accel;
    copyInfo.mode = VK_COPY_ACCELERATION_STRUCTURE_MODE_COMPACT_KHR;
    vkCmdCopyAccelerationStructureKHR(cmdBuf, &copyInfo);
    cleanupAS[idx] = m_blas[idx].as;
    m_blas[idx].as = as;
    ~~~
    
    Finally, it destroys the uncompacted BLASes.
    
    ~~~ cpp
    for(auto as : cleanupAS)
      m_alloc->destroy(as);
    ~~~


# Including Files and Matching Values Between C++ And GLSL

Currently, there are eight constants that must match between the C++ and GLSL code. These are the color buffer resolution:

~~~ cpp
static const uint64_t render_width     = 800;
static const uint64_t render_height    = 600;
~~~
[The C++ definition of the color buffer resolution.]

<br/>

~~~ glsl
  // The resolution of the buffer, which in this case is a hardcoded vector
  // of 2 unsigned integers:
  const uvec2 resolution = uvec2(800, 600);
~~~
[The GLSL definition of the color buffer resolution.]

The workgroup size:

~~~ cpp
static const uint32_t workgroup_width  = 16;
static const uint32_t workgroup_height = 8;
~~~
[The C++ definition of the workgroup size.]

<br/>

~~~ glsl
layout(local_size_x = 16, local_size_y = 8, local_size_z = 1) in;
~~~
[The GLSL definition of the workgroup size.]

and the bindings of each of the four resources:
~~~ cpp
descriptorSetContainer.addBinding(0, VK_DESCRIPTOR_TYPE_STORAGE_BUFFER, 1, VK_SHADER_STAGE_COMPUTE_BIT);
descriptorSetContainer.addBinding(1, VK_DESCRIPTOR_TYPE_ACCELERATION_STRUCTURE_KHR, 1, VK_SHADER_STAGE_COMPUTE_BIT);
descriptorSetContainer.addBinding(2, VK_DESCRIPTOR_TYPE_STORAGE_BUFFER, 1, VK_SHADER_STAGE_COMPUTE_BIT);
descriptorSetContainer.addBinding(3, VK_DESCRIPTOR_TYPE_STORAGE_BUFFER, 1, VK_SHADER_STAGE_COMPUTE_BIT);
...
writeDescriptorSets[0] = descriptorSetContainer.makeWrite(0 /*set index*/, 0 /*binding*/, &descriptorBufferInfo);
...
writeDescriptorSets[1] = descriptorSetContainer.makeWrite(0, 1, &descriptorAS);
..
writeDescriptorSets[2] = descriptorSetContainer.makeWrite(0, 2, &vertexDescriptorBufferInfo);
...
writeDescriptorSets[3] = descriptorSetContainer.makeWrite(0, 3, &indexDescriptorBufferInfo);
~~~
[Specifying binding locations in C++ using the second argument of `nvvk::DescriptorSetContainer::makeWrite`.]

<br/>

~~~ glsl highlight
layout(binding = 0, set = 0, scalar) buffer storageBuffer
~~~ glsl
{
  vec3 imageData[];
};
~~~ glsl highlight
layout(binding = 1, set = 0) uniform accelerationStructureEXT tlas;
layout(binding = 2, set = 0, scalar) buffer Vertices
~~~ glsl
{
  vec3 vertices[];
};
~~~ glsl highlight
layout(binding = 3, set = 0, scalar) buffer Indices
~~~ glsl
{
  uint indices[];
};
~~~
[Specifying binding locations in GLSL using the layout qualifier.]

These must match; otherwise, we could get an error from the validation layer, cause an access violation, or render incorrect data.

However, both C++ and GLSL can include files -- so in this chapter, we'll show how to write a file that can be included from both C++ and GLSL, and can define these constants in one place instead of two. In future chapters, we'll also use this file to share structures between the two languages.

!!! Tip
    You can find the reference code at the end of this chapter in the `checkpoints/e4_include` directory.
    If you run into any problems with the code in this chapter, you can diff your
    files against the reference files to see if there are any differences.

To start out, create a new file called `common.h` next to `main.cpp` in your project directory. This is the file that both C++ and GLSL will include. Add the following to `common.h`:

~~~ cpp
// Common file shared across C++ CPU code and GLSL GPU code.
#ifndef VK_MINI_PATH_TRACER_COMMON_H
#define VK_MINI_PATH_TRACER_COMMON_H

#define RENDER_WIDTH 800
#define RENDER_HEIGHT 600
#define WORKGROUP_WIDTH 16
#define WORKGROUP_HEIGHT 8

#define BINDING_IMAGEDATA 0
#define BINDING_TLAS 1
#define BINDING_VERTICES 2
#define BINDING_INDICES 3

#endif // #ifndef VK_MINI_PATH_TRACER_COMMON_H
~~~

This uses a standard C-style header guard in case this file gets included multiple times, and then defines the values of our eight constants. If you're using an IDE, it might make sense to reconfigure and regenerate the project from CMake at this point to make sure `common.h` shows up in your IDE, but the build system should work correctly with or without doing so.

We then need to replace each files' versions of these constants with a reference to the constant in `common.h`.

## Including `common.h` from C++

Let's start with the C++ code in `main.cpp`. Replace

~~~ cpp
static const uint64_t render_width     = 800;
static const uint64_t render_height    = 600;
static const uint32_t workgroup_width  = 16;
static const uint32_t workgroup_height = 8;
~~~

with

~~~ cpp
#include "common.h"
~~~

; replace

~~~ cpp
  VkDeviceSize       bufferSizeBytes  = render_width * render_height * 3 * sizeof(float);
~~~

with

~~~ cpp
  VkDeviceSize       bufferSizeBytes  = RENDER_WIDTH * RENDER_HEIGHT * 3 * sizeof(float);
~~~

; replace

~~~ cpp
  descriptorSetContainer.addBinding(0, VK_DESCRIPTOR_TYPE_STORAGE_BUFFER, 1, VK_SHADER_STAGE_COMPUTE_BIT);
  descriptorSetContainer.addBinding(1, VK_DESCRIPTOR_TYPE_ACCELERATION_STRUCTURE_KHR, 1, VK_SHADER_STAGE_COMPUTE_BIT);
  descriptorSetContainer.addBinding(2, VK_DESCRIPTOR_TYPE_STORAGE_BUFFER, 1, VK_SHADER_STAGE_COMPUTE_BIT);
  descriptorSetContainer.addBinding(3, VK_DESCRIPTOR_TYPE_STORAGE_BUFFER, 1, VK_SHADER_STAGE_COMPUTE_BIT);
~~~

with

~~~ cpp
  descriptorSetContainer.addBinding(BINDING_IMAGEDATA, VK_DESCRIPTOR_TYPE_STORAGE_BUFFER, 1, VK_SHADER_STAGE_COMPUTE_BIT);
  descriptorSetContainer.addBinding(BINDING_TLAS, VK_DESCRIPTOR_TYPE_ACCELERATION_STRUCTURE_KHR, 1, VK_SHADER_STAGE_COMPUTE_BIT);
  descriptorSetContainer.addBinding(BINDING_VERTICES, VK_DESCRIPTOR_TYPE_STORAGE_BUFFER, 1, VK_SHADER_STAGE_COMPUTE_BIT);
  descriptorSetContainer.addBinding(BINDING_INDICES, VK_DESCRIPTOR_TYPE_STORAGE_BUFFER, 1, VK_SHADER_STAGE_COMPUTE_BIT);
~~~

; replace

~~~ cpp
  // 0
  VkDescriptorBufferInfo descriptorBufferInfo{};
  descriptorBufferInfo.buffer = buffer.buffer;    // The VkBuffer object
  descriptorBufferInfo.range  = bufferSizeBytes;  // The length of memory to bind; offset is 0.
  writeDescriptorSets[0]      = descriptorSetContainer.makeWrite(0 /*set index*/, 0 /*binding*/, &descriptorBufferInfo);
  // 1
  VkWriteDescriptorSetAccelerationStructureKHR descriptorAS = nvvk::make<VkWriteDescriptorSetAccelerationStructureKHR>();
  VkAccelerationStructureKHR tlasCopy = raytracingBuilder.getAccelerationStructure();  // So that we can take its address
  descriptorAS.accelerationStructureCount = 1;
  descriptorAS.pAccelerationStructures    = &tlasCopy;
  writeDescriptorSets[1]                  = descriptorSetContainer.makeWrite(0, 1, &descriptorAS);
  // 2
  VkDescriptorBufferInfo vertexDescriptorBufferInfo{};
  vertexDescriptorBufferInfo.buffer = vertexBuffer.buffer;
  vertexDescriptorBufferInfo.range  = VK_WHOLE_SIZE;
  writeDescriptorSets[2]            = descriptorSetContainer.makeWrite(0, 2, &vertexDescriptorBufferInfo);
  // 3
  VkDescriptorBufferInfo indexDescriptorBufferInfo{};
  indexDescriptorBufferInfo.buffer = indexBuffer.buffer;
  indexDescriptorBufferInfo.range  = VK_WHOLE_SIZE;
  writeDescriptorSets[3]           = descriptorSetContainer.makeWrite(0, 3, &indexDescriptorBufferInfo);
~~~

with

~~~ cpp highlight
  // Color buffer
~~~ cpp
  VkDescriptorBufferInfo descriptorBufferInfo{};
  descriptorBufferInfo.buffer = buffer.buffer;    // The VkBuffer object
  descriptorBufferInfo.range  = bufferSizeBytes;  // The length of memory to bind; offset is 0.
~~~ cpp highlight
  writeDescriptorSets[0] = descriptorSetContainer.makeWrite(0 /*set index*/, BINDING_IMAGEDATA /*binding*/, &descriptorBufferInfo);
  // Top-level acceleration structure (TLAS)
~~~ cpp
  VkWriteDescriptorSetAccelerationStructureKHR descriptorAS = nvvk::make<VkWriteDescriptorSetAccelerationStructureKHR>();
  VkAccelerationStructureKHR tlasCopy = raytracingBuilder.getAccelerationStructure();  // So that we can take its address
  descriptorAS.accelerationStructureCount = 1;
  descriptorAS.pAccelerationStructures    = &tlasCopy;
~~~ cpp highlight
  writeDescriptorSets[1]                  = descriptorSetContainer.makeWrite(0, BINDING_TLAS, &descriptorAS);
  // Vertex buffer
~~~ cpp
  VkDescriptorBufferInfo vertexDescriptorBufferInfo{};
  vertexDescriptorBufferInfo.buffer = vertexBuffer.buffer;
  vertexDescriptorBufferInfo.range  = VK_WHOLE_SIZE;
~~~ cpp highlight
  writeDescriptorSets[2] = descriptorSetContainer.makeWrite(0, BINDING_VERTICES, &vertexDescriptorBufferInfo);
  // Index buffer
~~~ cpp
  VkDescriptorBufferInfo indexDescriptorBufferInfo{};
  indexDescriptorBufferInfo.buffer = indexBuffer.buffer;
  indexDescriptorBufferInfo.range  = VK_WHOLE_SIZE;
~~~ cpp highlight
  writeDescriptorSets[3]           = descriptorSetContainer.makeWrite(0, BINDING_INDICES, &indexDescriptorBufferInfo);
~~~

; replace

~~~ cpp
  vkCmdDispatch(cmdBuffer, (uint32_t(render_width) + workgroup_width - 1) / workgroup_width,
                (uint32_t(render_height) + workgroup_height - 1) / workgroup_height, 1);
~~~

with

~~~ cpp
  vkCmdDispatch(cmdBuffer, (uint32_t(RENDER_WIDTH) + WORKGROUP_WIDTH - 1) / WORKGROUP_WIDTH,
                (uint32_t(RENDER_HEIGHT) + WORKGROUP_HEIGHT - 1) / WORKGROUP_HEIGHT, 1);
~~~

; and replace

~~~ cpp
  stbi_write_hdr("out.hdr", render_width, render_height, 3, reinterpret_cast<float*>(data));
~~~

with

~~~ cpp
  stbi_write_hdr("out.hdr", RENDER_WIDTH, RENDER_HEIGHT, 3, reinterpret_cast<float*>(data));
~~~

## Including `common.h` from GLSL

GLSL uses the same `#include` preprocessor directive as C++, but we need to enable or require the `GL_GOOGLE_include_directive` extension to use it with `glslangValidator`. Since `raytrace.comp.glsl` is in the `shaders` subdirectory, we also make sure to include `../common.h` instead of `common.h`. Add the following lines to the `#extension` block of `raytrace.comp.glsl`:

~~~ glsl
#extension GL_GOOGLE_include_directive : require
#include "../common.h"
~~~

Now that we've included `../common.h`, replace

~~~ glsl
layout(local_size_x = 16, local_size_y = 8, local_size_z = 1) in;
~~~

with

~~~ glsl
layout(local_size_x = WORKGROUP_WIDTH, local_size_y = WORKGROUP_HEIGHT, local_size_z = 1) in;
~~~

;

~~~ glsl
layout(binding = 0, set = 0, scalar) buffer storageBuffer
{
  vec3 imageData[];
};
layout(binding = 1, set = 0) uniform accelerationStructureEXT tlas;
layout(binding = 2, set = 0, scalar) buffer Vertices
{
  vec3 vertices[];
};
layout(binding = 3, set = 0, scalar) buffer Indices
{
  uint indices[];
};
~~~

with

~~~ glsl highlight
layout(binding = BINDING_IMAGEDATA, set = 0, scalar) buffer storageBuffer
~~~ glsl
{
  vec3 imageData[];
};
~~~ glsl highlight
layout(binding = BINDING_TLAS, set = 0) uniform accelerationStructureEXT tlas;
layout(binding = BINDING_VERTICES, set = 0, scalar) buffer Vertices
~~~ glsl
{
  vec3 vertices[];
};
~~~ glsl highlight
layout(binding = BINDING_INDICES, set = 0, scalar) buffer Indices
~~~ glsl
{
  uint indices[];
};
~~~

; and

~~~ glsl
  const uvec2 resolution = uvec2(800, 600);
~~~

with

~~~ glsl
  const uvec2 resolution = uvec2(RENDER_WIDTH, RENDER_HEIGHT);
~~~

You should now be able to build and run the application, and it should run the same as before -- but now these constants are only defined in one place, and are easier to change if you ever decide to add new descriptors or change the image resolution or workgroup size.

!!! Tip Polyglot Files
    We've just written an example of a part of a [polyglot](https://en.wikipedia.org/wiki/Polyglot_%28computing%29) file -- a piece of code that's valid in multiple programming languages (in this case, C++ and GLSL). This will get a bit more complex in the next chapter, but C and GLSL are similar enough that this isn't a problem (although sometimes differences in the way C and GLSL pack structs can cause unexpected issues if one isn't aware of them; we'll discuss this in the next chapter).
    
    Since multiple languages can be compiled to SPIR-V, there are now projects like [Rust GPU](https://github.com/EmbarkStudios/rust-gpu) and [Circle C++ Shaders](https://github.com/seanbaxter/shaders/blob/master/README.md) that can compile code written in these programming languages to SPIR-V. In these cases, a file like `common.h` wouldn't be a polyglot -- all the code would be written in C++ or Rust, for instance!

!!! WARNING Shader Build System Limitations
    Note that this project's CMake system won't rebuild a GLSL shader if only its include files change. In these cases, it's necessary to rebuild the project, or to add some small change to the GLSL shader file containing `main()`.

# Push Constants

Now, we're still compiling the resolution of the color buffer directly into the SPIR-V shader code. Instead of having to recompile the shader every time the resolution changes, we can pass this information to the pipeline like an argument of a function.

There are multiple ways to pass this data to a pipeline. One way would be to
create a buffer, copy this data into the buffer, and then reference this buffer
in a descriptor. This is the idea behind **uniform buffers** -- which as opposed
to storage buffers, are a type of buffer descriptor that shaders can only read,
not write. Uniform buffers usually have a size limit of around 65,536 bytes.

!!! Tip
    The exact limit is `VkPhysicalDeviceLimits::maxUniformBufferRange`, which can be accessed using
    
    ~~~
    nvvk::Context::m_physicalInfo::properties10::limits::maxUniformBufferRange
    ~~~

For smaller amounts of data -- like two `uint`s for the resolution -- we can also use **push constants**, which is an excellent fit for this use case. The mental model is that push constants are a small amount of memory -- usually 256 bytes, although the minimum is 128 -- that shaders can read.

!!! Tip
    See `VkPhysicalDeviceLimits::maxPushConstantsSize` for the exact limit on the size of the push constants for your GPU.

The pipeline layout says which shader stages can access which ranges of the push constants using **push constant ranges**, and one can record commands to write values into the push constants by recording `vkCmdPushConstants` commands. When a command buffer starts, the contents of the push constants are initially undefined; then each `vkCmdPushConstants` call "pushes" new data to the push constants, overwriting previous values, and then all subsequent commands on that command buffer use the updated push constants.

GLSL shaders are able to access push constants using a `layout(push_constant)` block of variables. Although we won't see it in this tutorial, it's possible to use the `offset` parameter in variable layout qualifiers in GLSL to specify that a shader only uses a subset of the push constants, instead of the first range of the push constants.

So, in this chapter, we'll:

- Define a `struct` in `common.h` so that we don't need to define how the push constants are laid out twice;
- Add a push constant range to the pipeline layout (it's probably a bit easier to see why pipeline layouts are different than descriptor set layouts now: descriptor set layouts only include descriptors, while pipeline layouts include both descriptor set layouts and push constant ranges);
- Add a command to write the push constants;
- Modify the GLSL code to get the color buffer resolution from the push constants instead of from `RENDER_WIDTH` and `RENDER_HEIGHT`.

This winds up being similar to how we added descriptors in [the main tutorial](index.html#descriptors/listingandallocatingdescriptors). Let's get started!

!!! Tip
    You can find the reference code at the end of this chapter in the `checkpoints/e5_push_constants` directory.
    If you run into any problems with the code in this chapter, you can diff your
    files against the reference files to see if there are any differences.


## Adding a Struct to `common.h`

First, remove the following two lines from `common.h`, since we won't hardcode the width and height of the color buffer any more:

~~~ cpp
#define RENDER_WIDTH 800
#define RENDER_HEIGHT 600
~~~

Replace them with the following code, which describes a struct type that both C++ and GLSL code will use. This must be both valid C++ and valid GLSL code -- so since a `uint` is a 32-bit unsigned integer in GLSL, but `uint` isn't built into the C++ language, we add the appropriate type alias if this file is being used to compile C++ code:

~~~ cpp
#ifdef __cplusplus
#include <cstdint>
using uint = uint32_t;
#endif  // #ifdef __cplusplus

struct PushConstants
{
  uint render_width;
  uint render_height;
};
~~~

!!! Tip Struct Packing in GLSL
    By default, elements in structs in GLSL are padded according to `std140` rules (as defined in the "Standard Uniform Block Layout" section of the [OpenGL specification](https://www.khronos.org/registry/OpenGL/specs/gl/glspec46.core.pdf#page=168)). Informally (and ignoring arrays and matrices), `std140` uses a different packing than C++: extra space is added between elements so that they never cross a 16-byte boundary (the same as the size of a `vec4`, for historical reasons). This isn't an issue here, but often comes up when including `vec3`s in structures. Two solutions are to manually pad or rearrange arrays so that C++ and `std140` layout are identical, or to use the `scalar` layout (which packs more compactly). 


## Accessing Push Constants from GLSL

Add the following code to `raytrace.comp.glsl` after the definition of the `buffer Indices` and before the definition of `stepAndOutputRNGFloat`:

~~~ glsl
layout(push_constant) uniform PushConsts
{
  PushConstants pushConstants;
};
~~~

This defines a variable, `pushConstants`, which is a struct of type `PushConstants`, and starts at offset 0 of the push constants.

!!! Tip
    GLSL shaders can only have 0 or 1 blocks with the `push_constant` layout qualifier.

We can access the elements of `pushConstants` like accessing a struct in C++, using the dot operator. Replace the line

~~~ glsl
  // The resolution of the buffer, which in this case is a hardcoded vector
  // of 2 unsigned integers:
  const uvec2 resolution = uvec2(RENDER_WIDTH, RENDER_HEIGHT);
~~~

with

~~~ glsl
  // The resolution of the buffer, which in this case is a vector
  // of 2 unsigned integers:
  const uvec2 resolution = uvec2(pushConstants.render_width, pushConstants.render_height);
~~~


## Push Constants in Vulkan

We'll now add push constant ranges to the pipeline layout and add a command to write push constants.

In `main.cpp`, start by creating a `PushConstants` struct, as follows. Note that I've changed the resolution to 1024 x 600 from 800 x 600, so that it's easy to see that the push constants had an effect:

~~~ cpp
#include "common.h"

~~~ cpp highlight
PushConstants pushConstants{1024 /* render_width */, 600 /* render_height */};
~~~

Next, replace `RENDER_WIDTH` and `RENDER_HEIGHT` with `pushConstants.render_width` and `pushConstants.render_height` respectively when setting `bufferSizeBytes` by replacing

~~~ cpp
  VkDeviceSize       bufferSizeBytes  = RENDER_WIDTH * RENDER_HEIGHT * 3 * sizeof(float);
~~~

with

~~~ cpp
  VkDeviceSize bufferSizeBytes = uint64_t(pushConstants.render_width) * uint64_t(pushConstants.render_height) * 3 * sizeof(float);
~~~

We can create a pipeline layout with multiple push constant ranges. For this tutorial, we'll only have one `VkPushConstantRange`, making the `sizeof(PushConstants)` bytes starting at byte 0 of the push constants accessible to compute shaders. Since [the Vulkan specification for `VkPushConstantRange`](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VkPushConstantRange.html) requires the size of the range to be divisible by 4, we also add a static (compile-time) assertion for this.

Replace

~~~ cpp
  // Create a simple pipeline layout from the descriptor set layout:
  descriptorSetContainer.initPipeLayout();
~~~

with the following:

~~~ cpp
  // Create a push constant range describing the amount of data for the push constants.
  static_assert(sizeof(PushConstants) % 4 == 0, "Push constant size must be a multiple of 4 per the Vulkan spec!");
  VkPushConstantRange pushConstantRange;
  pushConstantRange.stageFlags = VK_SHADER_STAGE_COMPUTE_BIT;
  pushConstantRange.offset     = 0;
  pushConstantRange.size       = sizeof(PushConstants);
  // Create a pipeline layout from the descriptor set layout and push constant range:
  descriptorSetContainer.initPipeLayout(1,                    // Number of push constant ranges
                                        &pushConstantRange);  // Pointer to push constant ranges
~~~

That creates the new pipeline layout with the push constant ranges, and the rest of the descriptor setup works normally!

Next, add a command to push the contents of the `pushConstants` variable to
bytes `0` through `sizeof(PushConstants) - 1` of the push constants, visible to
the compute shader. (The `stageFlags` here are redundant, and must match the
`stageFlags` we used in the pipeline layout.) We'll also change the 
`vkCmdDispatch` call to use `pushConstants.render_width` and
`pushConstants.render_height` instead of `RENDER_WIDTH` and `RENDER_HEIGHT`.

Replace

~~~ cpp
  // Run the compute shader with enough workgroups to cover the entire buffer:
  vkCmdDispatch(cmdBuffer, (uint32_t(RENDER_WIDTH) + WORKGROUP_WIDTH - 1) / WORKGROUP_WIDTH,
                (uint32_t(RENDER_HEIGHT) + WORKGROUP_HEIGHT - 1) / WORKGROUP_HEIGHT, 1);
~~~

with the following:

~~~ cpp
  // Push push constants:
  vkCmdPushConstants(cmdBuffer,                               // Command buffer
                     descriptorSetContainer.getPipeLayout(),  // Pipeline layout
                     VK_SHADER_STAGE_COMPUTE_BIT,             // Stage flags
                     0,                                       // Offset
                     sizeof(PushConstants),                   // Size in bytes
                     &pushConstants);                         // Data

  // Run the compute shader with enough workgroups to cover the entire buffer:
  vkCmdDispatch(cmdBuffer, (pushConstants.render_width + WORKGROUP_WIDTH - 1) / WORKGROUP_WIDTH,
                (pushConstants.render_height + WORKGROUP_HEIGHT - 1) / WORKGROUP_HEIGHT, 1);
~~~

Finally, replace

~~~ cpp
  stbi_write_hdr("out.hdr", RENDER_WIDTH, RENDER_HEIGHT, 3, reinterpret_cast<float*>(data));
~~~

with

~~~ cpp
  stbi_write_hdr("out.hdr", pushConstants.render_width, pushConstants.render_height, 3, reinterpret_cast<float*>(data));
~~~

You should now be able to build and run the program, and get a path traced 1024 x 600 image! The command buffer wrote the `pushConstants` struct to the push constants, which was then read by the compute shader.

![The Cornell box rendered at a resolution of 1024 x 600.](images/e5-1024-600.png)



# More Samples

Averaging more samples per pixel will generally reduce the level of noise in a path traced image.

![1 sample per pixel.](images/e6-1spp.png)

![64 samples per pixel.](images/e6-64spp.png)

![2048 samples per pixel.](images/e6-2048spp.png)

However, on Windows systems, setting `NUM_SAMPLES` to a number that's too high and thus causes the command buffer to take too long to complete (usually two seconds) can cause Windows' [**Timeout Detection and Recovery (TDR)**](https://docs.microsoft.com/en-us/windows-hardware/drivers/display/timeout-detection-and-recovery) system to reset the graphics driver and make the command buffer terminate early!

The solution is to break up command buffers that can take longer than two seconds to execute into smaller command buffers. In this chapter, we'll show how to average multiple path traced images into a single image containing thousands of samples per pixel -- instead of the 64 spp (samples per pixel) we've been working with until now -- without risking TDR. We'll also add a progress indicator to the rendering.

At the end of this chapter, it's possible to measure that this application can path trace 2048 samples per pixel of an 800 x 600 image with a maximum path length of 32 in less than 0.7 seconds on an RTX 3090.

Here's some additional information on TDR, noise, and command buffer lengths:

!!! Tip TDR and Device Removal: How WDDM Handles the Halting Problem
    If an application puts a GPU into an infinite loop, the system might not be able to display new images. (This isn't necessarily true if the GPU is being used as a multi-instance GPU, or if a system's display compositor is running on a different GPU.) Before TDR existed, this would have required the user to restart the machine. However, because the [halting problem](https://en.wikipedia.org/wiki/Halting_problem) for GLSL code is undecidable, there's no way to determine if running a shader will cause the GPU to get stuck in an infinite loop.
    
    The [Timeout Detection and Recovery](https://docs.microsoft.com/en-us/windows-hardware/drivers/display/timeout-detection-and-recovery) system in WDDM (Windows Display Driver Model) handles this problem by resetting the driver stack if a GPU can't complete or preempt a particular task within a set amount of time after Windows' GPU scheduler tries to preempt that task. By default, this duration of time is 2 seconds, but it can be changed by [modifying the Windows registry](https://docs.microsoft.com/en-us/windows-hardware/drivers/display/tdr-registry-keys) (although note that having your application change the TDR setting is usually inadvisable). On Linux-based systems, there usually isn't an equivalent timeout as of this writing.
    
    When TDR occurs, Vulkan returns a `VK_ERROR_DEVICE_LOST` error. When that happens, the entire `VkPhysicalDevice` needs to be recreated. Vulkan extensions like [`VK_NV_diagnostic_checkpoints`](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VK_NV_device_diagnostic_checkpoints.html) and tools like [Nsight Aftermath](https://developer.nvidia.com/nsight-aftermath) can help pinpoint what caused device lost errors.

!!! Tip More Samples Might Not Be the Answer: Reducing Noise with Fewer Samples
    Many applications -- especially real-time applications -- can generate ray traced images with only one sample per pixel per frame or less (such as if each frame only samples every other pixel). This is both less noisy and faster than this tutorial's path tracer. Since this is a tutorial primarily for Vulkan, we talk about adding more samples because it's a good way to talk about TDR, and is also a common situation for offline path tracers.
    
    Here are a few techniques other than adding more samples that can be used to reduce noise and make ray traced applications faster!
    
    - Using a denoiser such as [NRD](https://developer.nvidia.com/nvidia-rt-denoiser) or the OptiX denoiser in the [`vk_denoise` sample](https://github.com/nvpro-samples/vk_denoise); denoisers have become faster and higher quality using techniques such as [ASVGF](https://github.com/NVIDIA/Q2RTX/blob/master/src/refresh/vkpt/shader/asvgf.glsl) and deep neural networks. Temporal denoisers use reprojection techniques to accumulate samples over multiple frames.
    - Rendering fewer pixels, and then upscaling to full resolution using an AI upscaler such as [DLSS](https://www.nvidia.com/en-us/geforce/technologies/dlss/).
    - Using a [stratified sampler](http://www.pbr-book.org/3ed-2018/Sampling_and_Reconstruction/Stratified_Sampling.html) such as [Halton](http://www.pbr-book.org/3ed-2018/Sampling_and_Reconstruction/The_Halton_Sampler.html) rather than a uniform pseudorandom number generator (what the path tracer currently uses). This can reduce noise when we take lots of samples -- using a [low-discrepancy sequence](https://en.wikipedia.org/wiki/Low-discrepancy_sequence) can make the amount of error in the image fall off like $1/n$ instead of $1/\sqrt{n}$, where $n$ is the number of samples.
    - Using [blue noise](https://blog.demofox.org/2018/01/30/what-the-heck-is-blue-noise/) instead of white noise (what we currently use) to make denoising easier. Blue noise has many applications and can be used without too much work. However, also see Eric Heitz and Laurent Belcour's paper [*Distributing Monte Carlo Errors as a Blue Noise in Screen Space by Permuting Pixel Seeds Between Frames*](https://eheitzresearch.wordpress.com/772-2/) as an example of a more sophisticated technique using blue noise.
    - Choosing sampling directions when bouncing off surfaces to better match the integrand of the rendering equation at a given point. There are many techniques that improve this, from multiple importance sampling to path guiding; see for instance Bitterli, Wyman, Pharr, Shirley, Lefohn, and Jarosz' paper [*Spatiotemporal reservoir resampling for real-time ray tracing with dynamic direct lighting*](https://cs.dartmouth.edu/wjarosz/publications/bitterli20spatiotemporal.html) as implemented in [RTXDI](https://developer.nvidia.com/rtxdi).
    
    It's also important to note that using this path tracer with `NUM_SAMPLES = 1` might be slower than expected, due to **shader divergence**: all threads within a warp (32 threads) of a work group must execute the same instructions (an execution mask keeps track of which invocations within a warp are active, making branching possible), so all invocations within a warp must wait for the slowest invocation to finish. Since each invocation can randomly trace between 1 and 32 rays (the limit on `tracedSegments`), this can be much slower than expected (this gets hidden when tracing 64 samples per pixel, because the relative variance in the number of rays each invocation traces is much less). Some ways of solving this problem are to limit the maximum number of traces per invocation to a smaller number, use an irradiance cache to simulate more bounces, use a wavefront path tracing approach, and to group similar rays (such as in Disney's Hyperion renderer).
    
!!! Tip How Long Should a Command Buffer Take?
    TDR limits command buffers on Windows to take less than 2 seconds. Because each command buffer takes time to start and end recording and for the driver to start it (usually on the order of tens of microseconds), we recommend aiming for between 15-30 command buffers and 5-10 `vkQueueSubmit()` calls (since `vkQueueSubmit` calls can launch multiple command buffers) per 16ms frame in our [*Vulkan Do's and Don'ts*](https://developer.nvidia.com/blog/vulkan-dos-donts/). In other words, avoid lots of tiny command buffers, and Windows will prevent you from running command buffers that are too long.

## Adding the Sample Batch Index to the Push Constants

!!! Tip
    You can find the reference code at the end of this chapter in the `checkpoints/e6_more_samples` directory.
    If you run into any problems with the code in this chapter, you can diff your
    files against the reference files to see if there are any differences.

We'll launch a series of command buffers, each averaging a batch of `NUM_SAMPLES` samples. Each time we launch the compute shader, it'll sample a different set of random numbers. Each invocation will then average its samples into a running average.

To start out, add the index of the sample batch to `PushConstants` in `common.h`:

~~~ cpp
struct PushConstants
{
  uint render_width;
  uint render_height;
~~~ cpp highlight
  uint sample_batch;
~~~ cpp
};
~~~

## Modifying GLSL Code

In `raytrace.comp.glsl`, seed the random number generator with a different number for each pixel and sample batch by replacing

~~~ glsl
  uint rngState = resolution.x * pixel.y + pixel.x;  // Initial seed
~~~

with the following:

~~~ glsl
  uint rngState = (pushConstants.sample_batch * resolution.y + pixel.y) * resolution.x + pixel.x;  // Initial seed
~~~

Before sample batch `N`, each element of `imageData` will contain the average of that pixel's colors from sample batches 0 through `N-1`:

$$ \text{imageData}[\text{linearIndex}] = \frac{c_0 + c_1 + \ldots + c_{N-1}}{N} $$

We need to update this to include the color from sample batch `N`:

$$ \text{imageData}'[\text{linearIndex}] = \frac{c_0 + c_1 + \ldots + c_{N-1} + c_N}{N+1} $$

To do this, we'll multiply the previous value of `imageData[linearIndex]` by `N`:

$$ \text{imageData}[\text{linearIndex}] * N = c_0 + c_1 + \ldots + c_{N-1} $$

add this sample batch's color:

$$ \text{imageData}[\text{linearIndex}] * N + c_N = c_0 + c_1 + \ldots + c_{N-1} + c_N $$

and then divide by `N+1`:

$$ \frac{\text{imageData}[\text{linearIndex}] * N + c_N}{N+1} = \frac{c_0 + c_1 + \ldots + c_{N-1} + c_N}{N+1} = \text{imageData}'[\text{linearIndex}] $$

Here's how to do that in GLSL. Replace the end of `main()` in `raytrace.comp.glsl` with the following:

~~~ glsl
  // Get the index of this invocation in the buffer:
~~~ glsl highlight
  uint linearIndex = resolution.x * pixel.y + pixel.x;
  // Blend with the averaged image in the buffer:
  vec3 averagePixelColor = summedPixelColor / float(NUM_SAMPLES);
  if(pushConstants.sample_batch != 0)
  {
    averagePixelColor =
        (pushConstants.sample_batch * imageData[linearIndex] + averagePixelColor) / (pushConstants.sample_batch + 1);
  }
  imageData[linearIndex] = averagePixelColor;
~~~ glsl
}
~~~

Note that we must handle a special case of floating-point arithmetic: when `sample_batch == 0`, `imageData[linearIndex]` hasn't been initialized, and might be `NaN` (although the GPU might happen to set it to 0 for us). In this case, `sample_batch * imageData[linearIndex] + averagePixelColor` would also be `NaN`! So in this case, we set `imageData[linearIndex]` to the average pixel color directly, and avoid reading the uninitialized value of `imageData[linearIndex]`.

## Launching Multiple Sample Batches

Our final changes for this chapter are in `main.cpp`. First, if you followed the previous chapter, change the resolution of the image back to 800 x 600 from 1024 x 600:

~~~ cpp
PushConstants pushConstants{800 /* render_width */, 600 /* render_height */};
~~~

Then replace the lines

~~~ cpp
  // Create and start recording a command buffer
  VkCommandBuffer cmdBuffer = AllocateAndBeginOneTimeCommandBuffer(context, cmdPool);
  ...
  // End and submit the command buffer, then wait for it to finish:
  EndSubmitWaitAndFreeCommandBuffer(context, context.m_queueGCT, cmdPool, cmdBuffer);
~~~

with the following loop over 32 sample batches:

~~~ cpp
  const uint32_t NUM_SAMPLE_BATCHES = 32;
  for(uint32_t sampleBatch = 0; sampleBatch < NUM_SAMPLE_BATCHES; sampleBatch++)
  {
    // Create and start recording a command buffer
    VkCommandBuffer cmdBuffer = AllocateAndBeginOneTimeCommandBuffer(context, cmdPool);

    // Bind the compute shader pipeline
    vkCmdBindPipeline(cmdBuffer, VK_PIPELINE_BIND_POINT_COMPUTE, computePipeline);
    // Bind the descriptor set
    VkDescriptorSet descriptorSet = descriptorSetContainer.getSet(0);
    vkCmdBindDescriptorSets(cmdBuffer, VK_PIPELINE_BIND_POINT_COMPUTE, descriptorSetContainer.getPipeLayout(), 0, 1,
                            &descriptorSet, 0, nullptr);

    // Push push constants:
    pushConstants.sample_batch = sampleBatch;
    vkCmdPushConstants(cmdBuffer,                               // Command buffer
                       descriptorSetContainer.getPipeLayout(),  // Pipeline layout
                       VK_SHADER_STAGE_COMPUTE_BIT,             // Stage flags
                       0,                                       // Offset
                       sizeof(PushConstants),                   // Size in bytes
                       &pushConstants);                         // Data

    // Run the compute shader with enough workgroups to cover the entire buffer:
    vkCmdDispatch(cmdBuffer, (pushConstants.render_width + WORKGROUP_WIDTH - 1) / WORKGROUP_WIDTH,
                  (pushConstants.render_height + WORKGROUP_HEIGHT - 1) / WORKGROUP_HEIGHT, 1);

    // On the last sample batch:
    if(sampleBatch == NUM_SAMPLE_BATCHES - 1)
    {
      // Add a command that says "Make it so that memory writes by the compute shader
      // are available to read from the CPU." (In other words, "Flush the GPU caches
      // so the CPU can read the data.") To do this, we use a memory barrier.
      // This is one of the most complex parts of Vulkan, so don't worry if this is
      // confusing! We'll talk about pipeline barriers more in the extras.
      VkMemoryBarrier memoryBarrier = nvvk::make<VkMemoryBarrier>();
      memoryBarrier.srcAccessMask   = VK_ACCESS_SHADER_WRITE_BIT;  // Make shader writes
      memoryBarrier.dstAccessMask   = VK_ACCESS_HOST_READ_BIT;     // Readable by the CPU
      vkCmdPipelineBarrier(cmdBuffer,                              // The command buffer
                           VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT,   // From the compute shader
                           VK_PIPELINE_STAGE_HOST_BIT,             // To the CPU
                           0,                                      // No special flags
                           1, &memoryBarrier,                      // An array of memory barriers
                           0, nullptr, 0, nullptr);                // No other barriers
    }

    // End and submit the command buffer, then wait for it to finish:
    EndSubmitWaitAndFreeCommandBuffer(context, context.m_queueGCT, cmdPool, cmdBuffer);

    nvprintf("Rendered sample batch index %d.\n", sampleBatch);
  }
~~~

Here, we've essentially placed the rendering code from last chapter into a loop. A few things to note:

- We average a total of `NUM_SAMPLE_BATCHES * NUM_SAMPLES = 32 * 64 = 2048` samples.
- We make sure to update `pushConstants.sample_batch` each time we render a different sample batch.
- We only perform the pipeline barrier making sure that memory writes by the compute shader are available to read from the GPU on the last sample batch.
- We print a line to the console for each sample batch, so that we can see the progress of the render.
- If we structured this in a different way, we could record the command buffer for the next sample batch while the GPU is running the current command buffer, overlapping CPU and GPU work. However, this didn't give a significant enough performance improvement to be included in this tutorial.

You should now be able to compile and run the project. The console output should look something like this:

~~~ output
 RT BLAS: reducing from: 6912 to: 2048 = 4864 (70.37% smaller)
Rendered sample batch index 0.
Rendered sample batch index 1.
Rendered sample batch index 2.
[...]
Rendered sample batch index 30.
Rendered sample batch index 31.
~~~

and `out.hdr` should now contain significantly less noise:

![`out.hdr` generated from this chapter's code.](images/e6-output.png)

If one analyzes the project in [Nsight Systems](https://developer.nvidia.com/nsight-systems) (see Section [Measuring Performance] for an Nsight Systems tutorial), the Timeline View should show each of the `vkQueueWaitIdle` events corresponding to a sample batch on a successful capture. On an RTX 3090, tracing all 2048 samples per pixel with a maximum ray depth of 32 at 800 x 600 takes about 640.87 ms, or about 0.313 ms per sample. (And as mentioned above, there are many ways to speed this up.)


# Images

[**`VkImage`**](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VkImage.html) (Vulkan image) objects are used to represent images, such as 2D color textures, and are one of the most common and versatile objects in Vulkan. However, because they can be a bit complex, we haven't mentioned them yet! In this chapter, we'll show how to use `VkImage`s and image memory barriers. This will also allow us to remove the render width and height from the push constants and access them from GLSL through the image properties instead.

## What Is an Image?

In Vulkan, an image can be much more than a two-dimensional, rectangular array of colors. Here are some different aspects of images in Vulkan, from [`VkImageCreateInfo`](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VkImageCreateInfo.html) and [`VkImageViewCreateInfo`](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VkImageViewCreateInfo.html):

- Images can be 1D, 2D, or 3D (think of a 1D, 2D, or 3D array of pixels).

- Images can have different [`VkFormat`s](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VkFormat.html) describing what data each pixel has and how it's stored. For instance, a texture can use the `VK_FORMAT_R8_UNORM` format to store a single 8-bit value per pixel, which can be one of the values {0, 1/255, 2/255, ..., 255/255} and is interpreted as the red channel. We'll use `VK_FORMAT_R32G32B32A32_SFLOAT` to store four (red, green, blue, and alpha) 32-bit signed floating-point values per pixel (16-bit floating-point is smaller, faster, and recommended, but this way we don't have to convert between the two when passing the data to `stbi_write_hdr`). There are many other types; for instance, compressed formats such as `VK_FORMAT_BC7_SRGB_BLOCK` can hold compressed images in GPU memory at full performance without ever having to compress them.

![A comparison of the memory requirements of an 8-bit per channel, 3-channel image, and an image compressed with BC7.](images/e7-compressimg-1536-montage.png)

- This means in particular that images can not only contain color data, but can also contain things like depth and stencil data, which are used in Z-buffers. Color data, depth data, and stencil data are some of the possible **aspects** of an image.

- Images can have [**mipmaps**](https://en.wikipedia.org/wiki/Mipmap) - a chain of smaller regions of memory for each image, each half the width and height of the next larger one. Usually, these are used to store lower-resolution filtered versions of a higher-resolution image. Using mipmaps in this way and blending between lower-resolution images when possible can both improve quality (by reducing aliasing) and improve performance.

![The level 0 mipmap of an image, followed by its level 1, 2, ... mipmaps correctly filtered, showing transparency over a checkerboard background. If the cube at the top-left isn't visible, click on the image to see it at full resolution. The total memory usage is about 4/3x the memory usage of the image without mipmapping.](images/e7-mipmapping-montage.png)

![Using mipmaps to avoid aliasing using one sample per pixel. Top: Sampling using only mip level 0. Middle: Using isotropic filtering, which can sample higher mip levels. Bottom: Using anisotropic filtering, which can also sample higher mip levels, but can average multiple samples. All filtering modes have hardware acceleration. From [Real-Time Rendering, 4th Edition](https://www.realtimerendering.com/), courtesy of [this page](https://www.realtimerendering.com/figures.html).](images/e7-RTR4.06.14.png)

- An image can be an **array** of multiple **layers**; each layer can have its own 1D, 2D, or 3D mipmapped data (although each layer must have the same number of mips). A mip of a layer of an image is called a **subresource** (of the image resource).

- Using **sparse images**, not all of an image and each of its subresources has to exist in memory at one time. This can be used to work with virtual textures hypothetically extending over many thousands of pixels in each direction, where only a subset of these pixels at each mip level contain data and take up space in memory.

![A sparse texture and four of its higher-level mips. From Cem Cebenoyan, [*Real Virtual Texturing - Taking Advantage of DirectX11.2 Tiled Resources*](https://developer.nvidia.com/sites/default/files/akamai/gameworks/events/gdc14/GDC_14_Real%20Virtual%20Texturing%20-%20Taking%20Advantage%20of%20DirectX%2011.2%20Tiled%20Resources.pdf).](images/e7-sparse.png)

- Images can be **multisampled**: each texel (texture element) in an image can contain color information for not only 1, but multiple samples.

- Images can [tile their data in different ways](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VkImageTiling.html) -- in addition to a row-major layout like a multidimensional C array (`VK_IMAGE_TILING_LINEAR`), they can also be laid out using `VK_IMAGE_TILING_OPTIMAL` tiling, which is a more complex way of ordering pixel data that is designed to be more optimal (generally by being more cache-friendly) for the GPU.

- Images can specify [in what ways they can be used](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VkImageUsageFlagBits.html), much like buffers.

- Images can have different [layouts](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VkImageLayout.html) such as `VK_IMAGE_LAYOUT_GENERAL` or `VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL`; many operations in the Vulkan API require images to be in a set of particular layouts. (As mentioned in the main tutorial, these layouts are ignored by NVIDIA GPU hardware as of this writing.)

- [`VkImageView`s](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VkImageView.html) can interpret a range of subresources of an image using a different format than they were created with, can enable cube mapping, and can remap (swizzle) the red, green, blue, and alpha components of an image.

- [`VkSampler`s](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VkSampler.html) won't be discussed further in this tutorial, but we include them for completeness. They specify settings for how images should be read -- including interpolation between pixels, filtering between mips, and wrapping (how to extend the image when a shader samples a texture with a coordinate outside the normal range of the image).

![Interpolating between pixels in an image using a sampler. Left: nearest-neighbor sampling. Middle: Linear sampling. Right: Cubic sampling. From [Real-Time Rendering, 4th Edition](https://www.realtimerendering.com/), courtesy of [this page](https://www.realtimerendering.com/figures.html).](images/e7-RTR4.06.08.png)

![Different sampler wrap modes. Left: `VK_SAMPLER_ADDRESS_MODE_REPEAT`. Middle left: `VK_SAMPLER_ADDRESS_MIRRORED_REPEAT`. Middle right: `VK_SAMPLER_ADDRESS_MODE_CLAMP_TO_EDGE`. Right: `VK_SAMPLER_ADDRESS_MODE_CLAMP_TO_BORDER` with a specified `borderColor`. From [Real-Time Rendering, 4th Edition](https://www.realtimerendering.com/), courtesy of [this page](https://www.realtimerendering.com/figures.html).](images/e7-RTR4.06.07.jpg)

We'll specify many of these settings when creating the images in this chapter. Impressively, the GPU has hardware accelerated support for all these possible combinations!


## Outline

Previously, we used one buffer for color data, which was accessible by both the CPU and GPU. We'll replace it with two images: we'll render to one, which will only be accessible by the GPU, then copy the data on the GPU from the first image to a second image which is accessible by both the GPU and CPU, then read the second image's data from the CPU. Both images will have one mip level, one array layer, and use the `VK_FORMAT_R32G32B32A32_SFLOAT` (four-channel, 32-bit, signed floating-point) format.

!!! Tip
    This adds a fourth channel, alpha, which is often used for opacity; this is because `VK_FORMAT_R32G32B32_SFLOAT` isn't supported, since GPUs [are optimized for 32-bit, 64-bit, and 128-bit memory loads](https://docs.nvidia.com/gameworks/content/developertools/desktop/analysis/report/cudaexperiments/sourcelevel/memorytransactions.htm). 
    
    If one's willing to perform 16-bit floating-point to 32-bit floating-point conversion for compatibility with `stbi_write_hdr`, note that `VK_FORMAT_R16G16B16A16_SFLOAT` will be half as large and potentially give 2x texturing performance.

More specifically, in the C++ code, we'll:

- Create two images and an image view.
 - The first image, `image`, will be used for accumulating samples, and will use `VK_IMAGE_TILING_OPTIMAL` tiling, the `VK_IMAGE_LAYOUT_UNDEFINED` layout, and the `VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT` memory property flags.
 - The second image, `imageLinear`, will be used for transferring, and will use `VK_IMAGE_TILING_LINEAR` tiling (so that the CPU can read it correctly), the `VK_IMAGE_LAYOUT_UNDEFINED` layout, and the `VK_MEMORY_PROPERTY_HOST_COHERENT_BIT`, `VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT`, and `VK_MEMORY_PROPERTY_HOST_CACHED_BIT` memory property flags.
 - The image view will point to the entirety of `image`.

!!! Tip
    `VkImage`s' initial layouts must be either `VK_IMAGE_LAYOUT_UNDEFINED` or `VK_IMAGE_LAYOUT_PREINITIALIZED` at the moment according to the documentation for [`VkImageCreateInfo`](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VkImageCreateInfo.html).

- **Transition the layouts** of `image` and `imageLinear` from `VK_IMAGE_LAYOUT_UNDEFINED` to `VK_IMAGE_LAYOUT_GENERAL` and from `VK_IMAGE_LAYOUT_UNDEFINED` to `VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL`, respectively.
 - We'll use an **image memory barrier** to do this.
 - Since pipeline barriers require a command buffer, we'll use the same command buffer used to upload data to `vertexBuffer` and `indexBuffer`.

- Replace the `BINDING_IMAGEDATA` storage buffer descriptor with a **storage image descriptor**, which supports both reading and writing. This is where we use the image view.

- Render the image.

- Transition `image`'s layout from `VK_IMAGE_LAYOUT_GENERAL` to `VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL`.

- Copy `image` to `imageLinear`.

- Add a pipeline barrier saying to make **transfer** writes (including copies) available to read from the CPU.

- Map `imageLinear` to the CPU.

- Write the HDR image from `imageLinear`'s data.

- Unmap `imageLinear`.

In the GLSL code, we'll:

- Replace the variable at binding `BINDING_IMAGEDATA` with an `rgba32f uniform image2D` image.

- Get the resolution of the image using the `imageSize` GLSL function.

- Replace reading and writing an array with loading and storing values in an image.

Let's get started!

!!! Tip
    You can find the reference code at the end of this chapter in the `checkpoints/e7_image` directory.
    If you run into any problems with the code in this chapter, you can diff your
    files against the reference files to see if there are any differences.

## Removing Width and Height from the Push Constants Struct

Instead of passing the width and height to the compute shader using push constants,
we can get the width and height of the color image directly from the image itself.

In `common.h`, rewrite `struct PushConstants` as follows:

~~~ cpp
struct PushConstants
{
  uint sample_batch;
};
~~~

Then in `main.cpp`, replace

~~~ cpp
PushConstants pushConstants{800 /* render_width */, 600 /* render_height */};
~~~

with

~~~ cpp
PushConstants  pushConstants;
const uint32_t render_width  = 800;
const uint32_t render_height = 600;
~~~


## Creating Images and an Image View

Let's start by including an additional header, `<nvvk/images_vk.hpp>`. This includes
the `nvvk::makeImageMemoryBarrier` function, which will make the synchronization
section later this chapter slightly faster:

~~~ cpp
#include <nvvk/error_vk.hpp>
~~~ cpp highlight
#include <nvvk/images_vk.hpp>
~~~ cpp
#include <nvvk/raytraceKHR_vk.hpp>        // For nvvk::RaytracingBuilderKHR
~~~

In `main.cpp`, remove the following code:

~~~ cpp
  // Create a buffer
  VkDeviceSize bufferSizeBytes = uint64_t(pushConstants.render_width) * uint64_t(pushConstants.render_height) * 3 * sizeof(float);
  VkBufferCreateInfo bufferCreateInfo = nvvk::make<VkBufferCreateInfo>();
  bufferCreateInfo.size               = bufferSizeBytes;
  bufferCreateInfo.usage              = VK_BUFFER_USAGE_STORAGE_BUFFER_BIT | VK_BUFFER_USAGE_TRANSFER_DST_BIT;
  // VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT means that the CPU can read this buffer's memory.
  // VK_MEMORY_PROPERTY_HOST_CACHED_BIT means that the CPU caches this memory.
  // VK_MEMORY_PROPERTY_HOST_COHERENT_BIT means that the CPU side of cache management
  // is handled automatically, with potentially slower reads/writes.
  nvvk::Buffer buffer = allocator.createBuffer(bufferCreateInfo,                         //
                                               VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT       //
                                                   | VK_MEMORY_PROPERTY_HOST_CACHED_BIT  //
                                                   | VK_MEMORY_PROPERTY_HOST_COHERENT_BIT);
~~~

Replace it with the following code to
- create an image, the `nvvk::Image image` using settings from a `VkImageCreateInfo` object;
- create an image view for all of `image`; and
- create `imageLinear` with linear tiling, different usage, and different memory properties.

We've included comments to describe what each line does.

~~~ cpp
  // Create an image. Images are more complex than buffers - they can have
  // multiple dimensions, different color+depth formats, be arrays of mips,
  // have multisampling, be tiled in memory in e.g. row-linear order or in an
  // implementation-dependent way (and this layout of memory can depend on
  // what the image is being used for), and be shared across multiple queues.
  // Here's how we specify the image we'll use:
  VkImageCreateInfo imageCreateInfo = nvvk::make<VkImageCreateInfo>();
  imageCreateInfo.imageType         = VK_IMAGE_TYPE_2D;
  // RGB32 images aren't usually supported, so we change this to a RGBA32 image.
  imageCreateInfo.format = VK_FORMAT_R32G32B32A32_SFLOAT;
  // Defines the size of the image:
  imageCreateInfo.extent = {render_width, render_height, 1};
  // The image is an array of length 1, and each element contains only 1 mip:
  imageCreateInfo.mipLevels   = 1;
  imageCreateInfo.arrayLayers = 1;
  // We aren't using MSAA (i.e. the image only contains 1 sample per pixel -
  // note that this isn't the same use of the word "sample" as in ray tracing):
  imageCreateInfo.samples = VK_SAMPLE_COUNT_1_BIT;
  // The driver controls the tiling of the image for performance:
  imageCreateInfo.tiling = VK_IMAGE_TILING_OPTIMAL;
  // This image is read and written on the GPU, and data can be transferred
  // from it:
  imageCreateInfo.usage = VK_IMAGE_USAGE_STORAGE_BIT | VK_IMAGE_USAGE_TRANSFER_SRC_BIT;
  // Image is only used by one queue:
  imageCreateInfo.sharingMode = VK_SHARING_MODE_EXCLUSIVE;
  // The image must be in either VK_IMAGE_LAYOUT_UNDEFINED or VK_IMAGE_LAYOUT_PREINITIALIZED
  // according to the specification; we'll transition the layout shortly,
  // in the same command buffer used to upload the vertex and index buffers:
  imageCreateInfo.initialLayout = VK_IMAGE_LAYOUT_UNDEFINED;
  nvvk::Image image             = allocator.createImage(imageCreateInfo);

  // Create an image view for the entire image
  // When we create a descriptor for the image, we'll also need an image view
  // that the descriptor will point to. This specifies what part of the image
  // the descriptor views, and how the descriptor views it.
  VkImageViewCreateInfo imageViewCreateInfo = nvvk::make<VkImageViewCreateInfo>();
  imageViewCreateInfo.image                 = image.image;
  imageViewCreateInfo.viewType              = VK_IMAGE_VIEW_TYPE_2D;
  imageViewCreateInfo.format                = imageCreateInfo.format;
  // We could use imageViewCreateInfo.components to make the components of the
  // image appear to be "swizzled", but we don't want to do that. Luckily,
  // all values are set to VK_COMPONENT_SWIZZLE_IDENTITY, which means
  // "don't change anything", by nvvk::make or zero initialization.
  // This says that the ImageView views the color part of the image (since
  // images can contain depth or stencil aspects):
  imageViewCreateInfo.subresourceRange.aspectMask = VK_IMAGE_ASPECT_COLOR_BIT;
  // This says that we only look at array layer 0 and mip level 0:
  imageViewCreateInfo.subresourceRange.baseArrayLayer = 0;
  imageViewCreateInfo.subresourceRange.layerCount     = 1;
  imageViewCreateInfo.subresourceRange.baseMipLevel   = 0;
  imageViewCreateInfo.subresourceRange.levelCount     = 1;
  VkImageView imageView;
  NVVK_CHECK(vkCreateImageView(context, &imageViewCreateInfo, nullptr, &imageView));

  // Also create an image using linear tiling that can be accessed from the CPU,
  // much like how we created the buffer in the main tutorial. The first image
  // will be entirely local to the GPU for performance, while this image can
  // be mapped to CPU memory. We'll copy data from the first image to this
  // image in order to read the image data back on the CPU.
  // As before, we'll transition the image layout in the same command buffer
  // used to upload the vertex and index buffers.
  imageCreateInfo.tiling = VK_IMAGE_TILING_LINEAR;
  imageCreateInfo.usage  = VK_IMAGE_USAGE_TRANSFER_DST_BIT;
  nvvk::Image imageLinear = allocator.createImage(imageCreateInfo,                           //
                                                  VK_MEMORY_PROPERTY_HOST_COHERENT_BIT       //
                                                      | VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT  //
                                                      | VK_MEMORY_PROPERTY_HOST_CACHED_BIT);
~~~


## Synchronization, Part 3: Layout Transitions and Image Memory Barriers

In the main tutorial, we talked about [global memory barriers](index.html#commands/commandbuffersandpools/synchronization,part1:apipelinebarrier), which are a type of pipeline barrier. (Recall that pipeline barriers synchronize memory; since you're reading the extra chapters, it may make sense to look at some of the sidebars going more in-depth into pipeline barriers in general in that chapter at this point.)

Image memory barriers are another kind of memory barrier. Like global memory barriers, they can express **execution dependencies** and **memory dependencies** (see the sidebars in the main tutorial for an explanation of these terms). However, image memory barriers can also:

- Transition the layout of an image from an old layout (such as `VK_IMAGE_LAYOUT_UNDEFINED`) to a new layout (such as `VK_IMAGE_LAYOUT_GENERAL`).
- Transfer ownership of an image between two different queues (such as from a graphics, compute, and transfer queue to a transfer queue).
- Apply this to only a range of subresources (individual mipmaps within layers; these images only have 1 subresource) of an image.

In this case, we want to transition the layouts of two images, so we'll need two `VkImageMemoryBarriers`:

- We'll need to make `image` accessible to shader reads and writes, and transition it from `VK_IMAGE_LAYOUT_UNDEFINED` to `VK_IMAGE_LAYOUT_GENERAL`.
- We'll need to make `imageLinear` accessible to transfer writes (since we'll copy `image` to it later), and transition it from `VK_IMAGE_LAYOUT_UNDEFINED` to `VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL`.

We'll use two utilities to make writing these image layout transitions simpler. First, `nvvk::makeAccessMaskPipelineStageFlags` can take access masks, and figure out what pipeline stages they're part of:

~~~ cpp
VkPipelineStageFlags exampleFlags = nvvk::makeAccessMaskPipelineStageFlags(VK_ACCESS_TRANSFER_WRITE_BIT);
// exampleFlags is now equal to VK_PIPELINE_STAGE_TRANSFER_BIT
// (nvvk::makeAccessMaskPipelineStageFlags can also handle bitwise
// ORs of flags)
~~~

and `nvvk::makeImageMemoryBarrier` can make an image memory barrier for an entire image without a queue ownership transfer. It takes a `VkImage`, source and destination access flags, source and destination layouts, and the aspects of an image (used in the subresource range).

We'll need to add the image memory barriers used to do the layout transitions to a command buffer. As it happens, `uploadCmdBuffer`, used to upload data to `vertexBuffer` and `indexBuffer`, is in the perfect place. Replace that block with the following code, with lines to add highlighted, and comments explaining what we're doing:

~~~ cpp
  // Upload the vertex and index buffers to the GPU.
  nvvk::Buffer vertexBuffer, indexBuffer;
  {
    // Start a command buffer for uploading the buffers
    VkCommandBuffer uploadCmdBuffer = AllocateAndBeginOneTimeCommandBuffer(context, cmdPool);
    // We get these buffers' device addresses, and use them as storage buffers and build inputs.
    const VkBufferUsageFlags usage = VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT | VK_BUFFER_USAGE_STORAGE_BUFFER_BIT
                                     | VK_BUFFER_USAGE_ACCELERATION_STRUCTURE_BUILD_INPUT_READ_ONLY_BIT_KHR;
    vertexBuffer = allocator.createBuffer(uploadCmdBuffer, objVertices, usage);
    indexBuffer  = allocator.createBuffer(uploadCmdBuffer, objIndices, usage);

~~~ cpp highlight
    // Also, let's transition the layout of `image` to `VK_IMAGE_LAYOUT_GENERAL`,
    // and the layout of `imageLinear` to `VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL`.
    // Although we use `imageLinear` later, we're transferring its layout as
    // early as possible. For more complex applications, tracking images and
    // operations using a graph is a good way to handle these types of images
    // automatically. However, for this tutorial, we'll show how to write
    // image transitions by hand.

    // To do this, we combine both transitions in a single pipeline barrier.
    // This pipeline barrier will say "Make it so that all writes to memory by
    const VkAccessFlags srcAccesses = 0;  // (since image and imageLinear aren't initially accessible)
    // finish and can be read correctly by
    const VkAccessFlags dstImageAccesses       = VK_ACCESS_SHADER_READ_BIT | VK_ACCESS_SHADER_WRITE_BIT;  // for image
    const VkAccessFlags dstImageLinearAccesses = VK_ACCESS_TRANSFER_WRITE_BIT;  // for imageLinear
    // "

    // Here's how to do that:
    const VkPipelineStageFlags srcStages = nvvk::makeAccessMaskPipelineStageFlags(srcAccesses);
    const VkPipelineStageFlags dstStages = nvvk::makeAccessMaskPipelineStageFlags(dstImageAccesses | dstImageLinearAccesses);
    VkImageMemoryBarrier imageBarriers[2];
    // Image memory barrier for `image` from UNDEFINED to GENERAL layout:
    imageBarriers[0] = nvvk::makeImageMemoryBarrier(image.image,                    // The VkImage
                                                    srcAccesses, dstImageAccesses,  // Source and destination access masks
                                                    VK_IMAGE_LAYOUT_UNDEFINED, VK_IMAGE_LAYOUT_GENERAL,  // Source and destination layouts
                                                    VK_IMAGE_ASPECT_COLOR_BIT);  // Aspects of an image (color, depth, etc.)
    // Image memory barrier for `imageLinear` from UNDEFINED to TRANSFER_DST_OPTIMAL layout:
    imageBarriers[1] = nvvk::makeImageMemoryBarrier(imageLinear.image,                    // The VkImage
                                                    srcAccesses, dstImageLinearAccesses,  // Source and destination access masks
                                                    VK_IMAGE_LAYOUT_UNDEFINED, VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL,  // Source and dst layouts
                                                    VK_IMAGE_ASPECT_COLOR_BIT);  // Aspects of an image (color, depth, etc.)
    // Include the two image barriers in the pipeline barrier:
    vkCmdPipelineBarrier(uploadCmdBuffer,       // The command buffer
                         srcStages, dstStages,  // Src and dst pipeline stages
                         0,                     // Flags for memory dependencies
                         0, nullptr,            // Global memory barrier objects
                         0, nullptr,            // Buffer memory barrier objects
                         2, imageBarriers);     // Image barrier objects

~~~ cpp
    EndSubmitWaitAndFreeCommandBuffer(context, context.m_queueGCT, cmdPool, uploadCmdBuffer);
    allocator.finalizeAndReleaseStaging();
  }
~~~

!!! Tip Image Memory Barriers Without Queue Family Ownership Transfers
    If an image memory barrier's `srcQueueFamilyIndex` and `dstQueueFamilyIndex` members are equal, Vulkan doesn't perform an ownership transfer. (In other words, it's not necessary to set both to `VK_QUEUE_FAMILY_IGNORED`.)

## Modifying the Descriptor Set

Now, we'll replace the storage buffer descriptor with an image buffer descriptor. Replace the `addBinding` calls as follows (we've highlighted the modified lines again):

~~~ cpp
  // Here's the list of bindings for the descriptor set layout, from raytrace.comp.glsl:
~~~ cpp highlight
  // 0 - a storage image (the image `image`)
~~~ cpp
  // 1 - an acceleration structure (the TLAS)
  // 2 - a storage buffer (the vertex buffer)
  // 3 - a storage buffer (the index buffer)
  nvvk::DescriptorSetContainer descriptorSetContainer(context);
~~~ cpp highlight
  descriptorSetContainer.addBinding(BINDING_IMAGEDATA, VK_DESCRIPTOR_TYPE_STORAGE_IMAGE, 1, VK_SHADER_STAGE_COMPUTE_BIT);
~~~ cpp
  descriptorSetContainer.addBinding(BINDING_TLAS, VK_DESCRIPTOR_TYPE_ACCELERATION_STRUCTURE_KHR, 1, VK_SHADER_STAGE_COMPUTE_BIT);
  descriptorSetContainer.addBinding(BINDING_VERTICES, VK_DESCRIPTOR_TYPE_STORAGE_BUFFER, 1, VK_SHADER_STAGE_COMPUTE_BIT);
  descriptorSetContainer.addBinding(BINDING_INDICES, VK_DESCRIPTOR_TYPE_STORAGE_BUFFER, 1, VK_SHADER_STAGE_COMPUTE_BIT);
~~~

We now need to modify the code that writes descriptors in the descriptor set. Descriptors point to resources -- and in the case of storage images, descriptors use image views, which point to a range of subresources.

Replace index 0 of `writeDescriptorSets` with a `VkWriteDescriptorSet` that says to make the descriptor point to `imageView`, and that the image will have layout `VK_IMAGE_LAYOUT_GENERAL`:

~~~ cpp
  // Color image
  VkDescriptorImageInfo descriptorImageInfo{};
  descriptorImageInfo.imageLayout = VK_IMAGE_LAYOUT_GENERAL;  // The image's layout
  descriptorImageInfo.imageView   = imageView;                // How the image should be accessed
  writeDescriptorSets[0] = descriptorSetContainer.makeWrite(0 /*set index*/, BINDING_IMAGEDATA /*binding*/, &descriptorImageInfo);
~~~

## Modifying the Dispatch

Since the push constants no longer contain `render_width` and `render_height`, change the `vkCmdDispatch` line to

~~~ cpp
    vkCmdDispatch(cmdBuffer, (render_width + WORKGROUP_WIDTH - 1) / WORKGROUP_WIDTH,
                  (render_height + WORKGROUP_HEIGHT - 1) / WORKGROUP_HEIGHT, 1);
~~~

## Copying Images

In the last sample batch command buffer, we'll add commands to transition `image` from `VK_IMAGE_LAYOUT_GENERAL` to `VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL` and make it so that it can be accessed for copying it to another image.
Then we'll copy the entire region of `image` to `imageLinear` (note that the two images have different tiling).
Finally, we'll issue the same pipeline barrier as from the main tutorial, but with the difference that we're now
making transfer writes readable by the CPU, instead of shader writes.

To copy one part of an image to a part of another image, we'll use [vkCmdCopyImage](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/vkCmdCopyImage.html). The
`VkImageCopy* pRegions` points to an array of copies of regions to perform -- but in this case, we'll only copy
one region, the entire image.

!!! Tip Copying, Blitting, and Resolving
    Vulkan has three similar-sounding commands: [vkCmdCopyImage](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/vkCmdCopyImage.html), [vkCmdBlitImage](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/vkCmdBlitImage.html), and [vkCmdResolveImage](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/vkCmdResolveImage.html), all of which are useful. There are a few differences between these.
    
    `vkCmdResolveImage` resolves a multisampled image to an image with
    one sample per pixel. (These are like smaller pixels with colors per pixel, not the samples we take during path tracing.)
    
    `vkCmdBlitImage` can resize a region when it copies it from one subresource to another. This can include
    texture interpolation using nearest-neighbor, linear, or cubic (if supported) filtering. It will also
    convert between formats, but it can't handle multisampled images.
    
    `vkCmdCopyImage` can't perform rescaling, and the formats must be [compatible](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/html/vkspec.html#formats-compatibility) (in most cases, this means that each pixel's data takes up the same number of bits; see the linked table for a full specification). This doesn't perform format conversion, so it can be used as one way to interpret data of one format as data of another format.

Replace the code

~~~ cpp
       // Add a command that says "Make it so that memory writes by the compute shader
      // are available to read from the CPU." (In other words, "Flush the GPU caches
      // so the CPU can read the data.") To do this, we use a memory barrier.
      // This is one of the most complex parts of Vulkan, so don't worry if this is
      // confusing! We'll talk about pipeline barriers more in the extras.
      VkMemoryBarrier memoryBarrier = nvvk::make<VkMemoryBarrier>();
      memoryBarrier.srcAccessMask   = VK_ACCESS_SHADER_WRITE_BIT;  // Make shader writes
      memoryBarrier.dstAccessMask   = VK_ACCESS_HOST_READ_BIT;     // Readable by the CPU
      vkCmdPipelineBarrier(cmdBuffer,                              // The command buffer
                           VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT,   // From the compute shader
                           VK_PIPELINE_STAGE_HOST_BIT,             // To the CPU
                           0,                                      // No special flags
                           1, &memoryBarrier,                      // An array of memory barriers
                           0, nullptr, 0, nullptr);
~~~

with the following (the first transition ensures that the ray tracing results
are visible and available to the upcoming copy command, and then the second
transition ensures that the copied data is visible and available to the CPU.)

~~~ cpp
      // Transition `image` from GENERAL to TRANSFER_SRC_OPTIMAL layout. See the
      // code for uploadCmdBuffer above to see a description of what this does:
      const VkAccessFlags        srcAccesses = VK_ACCESS_SHADER_READ_BIT | VK_ACCESS_SHADER_WRITE_BIT;
      const VkAccessFlags        dstAccesses = VK_ACCESS_TRANSFER_READ_BIT;
      const VkPipelineStageFlags srcStages   = nvvk::makeAccessMaskPipelineStageFlags(srcAccesses);
      const VkPipelineStageFlags dstStages   = nvvk::makeAccessMaskPipelineStageFlags(dstAccesses);
      const VkImageMemoryBarrier barrier =
          nvvk::makeImageMemoryBarrier(image.image,               // The VkImage
                                       srcAccesses, dstAccesses,  // Src and dst access masks
                                       VK_IMAGE_LAYOUT_GENERAL, VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL,  // Src and dst layouts
                                       VK_IMAGE_ASPECT_COLOR_BIT);
      vkCmdPipelineBarrier(cmdBuffer,             // Command buffer
                           srcStages, dstStages,  // Src and dst pipeline stages
                           0,                     // Dependency flags
                           0, nullptr,            // Global memory barriers
                           0, nullptr,            // Buffer memory barriers
                           1, &barrier);          // Image memory barriers

      // Now, copy the image (which has layout TRANSFER_SRC_OPTIMAL) to imageLinear
      // (which has layout TRANSFER_DST_OPTIMAL).
      {
        VkImageCopy region;
        // We copy the image aspect, layer 0, mip 0:
        region.srcSubresource.aspectMask     = VK_IMAGE_ASPECT_COLOR_BIT;
        region.srcSubresource.baseArrayLayer = 0;
        region.srcSubresource.layerCount     = 1;
        region.srcSubresource.mipLevel       = 0;
        // (0, 0, 0) in the first image corresponds to (0, 0, 0) in the second image:
        region.srcOffset      = {0, 0, 0};
        region.dstSubresource = region.srcSubresource;
        region.dstOffset      = {0, 0, 0};
        // Copy the entire image:
        region.extent = {render_width, render_height, 1};
        vkCmdCopyImage(cmdBuffer,                             // Command buffer
                       image.image,                           // Source image
                       VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL,  // Source image layout
                       imageLinear.image,                     // Destination image
                       VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL,  // Destination image layout
                       1, &region);                           // Regions
      }

      // Add a command that says "Make it so that memory writes by transfers
      // are available to read from the CPU." (In other words, "Flush the GPU caches
      // so the CPU can read the data.") To do this, we use a memory barrier.
      VkMemoryBarrier memoryBarrier = nvvk::make<VkMemoryBarrier>();
      memoryBarrier.srcAccessMask   = VK_ACCESS_TRANSFER_WRITE_BIT;  // Make transfer writes
      memoryBarrier.dstAccessMask   = VK_ACCESS_HOST_READ_BIT;       // Readable by the CPU
      vkCmdPipelineBarrier(cmdBuffer,                                // The command buffer
                           VK_PIPELINE_STAGE_TRANSFER_BIT,           // From transfers
                           VK_PIPELINE_STAGE_HOST_BIT,               // To the CPU
                           0,                                        // No special flags
                           1, &memoryBarrier,                        // An array of memory barriers
                           0, nullptr, 0, nullptr); 
~~~

## Mapping, Reading, and Unmapping

Mapping an image's memory works much like mapping a buffer's memory. Replace

~~~ cpp
  void* data = allocator.map(buffer);
  stbi_write_hdr("out.hdr", pushConstants.render_width, pushConstants.render_height, 3, reinterpret_cast<float*>(data));
  allocator.unmap(buffer);
~~~

with

~~~ cpp
  void* data = allocator.map(imageLinear);
  stbi_write_hdr("out.hdr", render_width, render_height, 4, reinterpret_cast<float*>(data));
  allocator.unmap(imageLinear);
~~~

Note especially that we've changed the fourth component of `stbi_write_hdr` -- the channels per pixel -- from 3 to 4! It's important to get this right; otherwise, the image will look like [the glitch from not using `scalar`](index.html#descriptors/cleanerbufferwritingusingscalarblocklayout) in the main tutorial.

## Cleaning Up

To finish the C++ code, replace

~~~ cpp
  allocator.destroy(buffer);
~~~

with

~~~ cpp
  allocator.destroy(imageLinear);
  vkDestroyImageView(context, imageView, nullptr);
  allocator.destroy(image);
~~~

## Images in GLSL

In `raytrace.comp.glsl`, replace the block of `layout` definitions as follows. This replaces the `storageBuffer` buffer with an `image2D storageImage`. It's required to have the `uniform` type qualifier according to the GLSL specification, and we say that the image uses the `rgba32f` format in the layout qualifier.

~~~ glsl highlight
// Binding BINDING_IMAGEDATA in set 0 is a storage image with four 32-bit floating-point channels,
// defined using a uniform image2D variable.
layout(binding = BINDING_IMAGEDATA, set = 0, rgba32f) uniform image2D storageImage;
layout(binding = BINDING_TLAS, set = 0) uniform accelerationStructureEXT tlas;
// The scalar layout qualifier here means to align types according to the alignment
// of their scalar components, instead of e.g. padding them to std140 rules.
~~~ glsl
layout(binding = BINDING_VERTICES, set = 0, scalar) buffer Vertices
{
  vec3 vertices[];
};
layout(binding = BINDING_INDICES, set = 0, scalar) buffer Indices
{
  uint indices[];
};
~~~

Next, we can get the resolution of the image from the image itself, instead of from the push constants. Image coordinates in GLSL are `ivec2`s instead of `uvec2`s, so we change the type of `resolution` to reduce the number of typecasts later in the file.

Replace

~~~ glsl
  // The resolution of the buffer, which in this case is a hardcoded vector
  // of 2 unsigned integers:
  const uvec2 resolution = uvec2(pushConstants.render_width, pushConstants.render_height);
~~~

with

~~~ glsl
  // The resolution of the image:
  const ivec2 resolution = imageSize(storageImage);
~~~

We do need a cast to get the `rngState` as a `uint` now, though:

~~~ glsl
  // State of the random number generator with an initial seed.
  uint rngState = uint((pushConstants.sample_batch * resolution.y + pixel.y) * resolution.x + pixel.x);
~~~

Finally, we can use the two functions

~~~ glsl
gvec4 imageLoad(gimage2D image,
 	              ivec2 P);
~~~

and

~~~ glsl
void imageStore(gimage2D image,
                ivec2 P,
                gvec4 data);
~~~

to load and store pixels of data from an `image2D` object. `P` is the index of the pixel whose data the function loads or stores. (GLSL uses 'g' to sort of denote a generic prefix; in this case, 'g' is empty, as we use an `image2D` and load and store a `vec4`.)

To use it, replace the previous code at the end of `raytrace.comp.glsl` for loading and storing the average from and new average color to a buffer: 

~~~ glsl
  // Get the index of this invocation in the buffer:
  uint linearIndex = resolution.x * pixel.y + pixel.x;
  // Blend with the averaged image in the buffer:
  vec3 averagePixelColor = summedPixelColor / float(NUM_SAMPLES);
  if(pushConstants.sample_batch != 0)
  {
    averagePixelColor =
        (pushConstants.sample_batch * imageData[linearIndex] + averagePixelColor) / (pushConstants.sample_batch + 1);
  }
  imageData[linearIndex] = averagePixelColor;
~~~

with the following code for doing the same with `storageImage`:

~~~ glsl
  // Blend with the averaged image in the buffer:
  vec3 averagePixelColor = summedPixelColor / float(NUM_SAMPLES);
  if(pushConstants.sample_batch != 0)
  {
    // Read the storage image:
    const vec3 previousAverageColor = imageLoad(storageImage, pixel).rgb;
    // Compute the new average:
    averagePixelColor =
        (pushConstants.sample_batch * previousAverageColor + averagePixelColor) / (pushConstants.sample_batch + 1);
  }
  // Set the color of the pixel `pixel` in the storage image to `averagePixelColor`:
  imageStore(storageImage, pixel, vec4(averagePixelColor, 0.0));
~~~

You should now be able to compile and run the application!



# Debug Names

If you run a debug build of the application, the Vulkan validation layer will warn you of incorrect API usage. Here's an example of a validation layer error, where I incorrectly forgot to transition one of the two images. I've reformatted it to make it easier to read:

~~~ output linenumbers
ERROR: UNASSIGNED-CoreValidation-DrawState-InvalidImageLayout
 --> Validation Error: [ UNASSIGNED-CoreValidation-DrawState-InvalidImageLayout ]
 Object 0: handle = 0x23396d6a788,
 type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x4dae5635 |
 Submitted command buffer expects VkImage 0x6dc7200000000005[]
 (subresource: aspectMask 0x1 array layer 0, mip level 0) to be in layout
 VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL--instead, current layout is
 VK_IMAGE_LAYOUT_UNDEFINED.
~~~

This tells me that I've missed a layout transition or incorrectly typed in an image layout somewhere -- now I have the problem of determining what object this error is talking about. There are a few ways I could determine this with knowledge of the codebase (I could observe that `imageLinear.image` has value `0x6dc7200000000005`, or that `imageLinear` is the only image that uses the `VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL` layout at some point) -- but one thing that can make this easier is to have the validation layer print out the name of the object!

Using the tools in this chapter, I can have the validation layer print out the following instead:

~~~ output linenumbers
ERROR: UNASSIGNED-CoreValidation-DrawState-InvalidImageLayout
 --> Validation Error: [ UNASSIGNED-CoreValidation-DrawState-InvalidImageLayout ]
 Object 0: handle = 0x174e9ac4848,
 type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x4dae5635 |
 Submitted command buffer expects VkImage 0x6dc7200000000005[imageLinear]
 (subresource: aspectMask 0x1 array layer 0, mip level 0) to be in layout
 VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL--instead, current layout is
 VK_IMAGE_LAYOUT_UNDEFINED.
~~~

The end of line 5 then tells me immediately that `imageLinear` does not have the correct layout, which helps me determine that I've forgotten to add a layout transition for `imageLinear` from `VK_IMAGE_LAYOUT_UNDEFINED` to `VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL`.

!!! Tip
    You can determine what line a validation layer error comes from by placing a breakpoint on the following line in `nvvk::Context::debugMessengerCallback`:
    
    ~~~ cpp
    nvprintfLevel(level, "ERROR: %s \n --> %s\n", callbackData->pMessageIdName, callbackData->pMessage);
    ~~~
    
    This will break any time the validation layer reports an error, and then one can look at the stack trace to see where the error came from.

To give an object a name, we'll use `vkSetDebugUtilsObjectNameEXT` through the `nvvk::DebugUtil` class. `nvvk::DebugUtil::setObjectName`
reduces creating a `VkDebugUtilsObjectNameInfoEXT` struct and calling `vkSetDebugUtilsObjectNameEXT` to a single line of code,
but one can also create their own version of `DebugUtil` or call `vkSetDebugUtilsObjectNameEXT` directly.

!!! Tip
    You can find the reference code at the end of this chapter in the `checkpoints/e8_debug_names` directory.
    If you run into any problems with the code in this chapter, you can diff your
    files against the reference files to see if there are any differences.

## Initializing `DebugUtil`

To create and initialize an `nvvk::DebugUtil` object, add the following line just before creating the `nvvk::ResourceAllocatorDedicated`, where we pass the device to the `DebugUtil` constructor.

~~~ cpp
  // Initialize the debug utilities:
  nvvk::DebugUtil debugUtil(context);
~~~

## Setting Object Names

The next step is to use `debugUtil.setObjectName` to set the names of all the objects we create. I've highlighted all the places I added `setObjectName` calls below:

`Image`:

~~~ cpp
  nvvk::Image image             = allocator.createImage(imageCreateInfo);
~~~ cpp highlight
  debugUtil.setObjectName(image.image, "image");
~~~

`ImageView`:

~~~ cpp
  NVVK_CHECK(vkCreateImageView(context, &imageViewCreateInfo, nullptr, &imageView));
~~~ cpp highlight
  debugUtil.setObjectName(imageView, "imageView");
~~~

`ImageLinear`:

~~~ cpp
  nvvk::Image imageLinear = allocator.createImage(imageCreateInfo,                           //
                                                  VK_MEMORY_PROPERTY_HOST_COHERENT_BIT       //
                                                      | VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT  //
                                                      | VK_MEMORY_PROPERTY_HOST_CACHED_BIT);
~~~ cpp highlight
  debugUtil.setObjectName(imageLinear.image, "imageLinear");
~~~

`CmdPool`:

~~~ cpp
  NVVK_CHECK(vkCreateCommandPool(context, &cmdPoolInfo, nullptr, &cmdPool));
~~~ cpp highlight
  debugUtil.setObjectName(cmdPool, "cmdPool");
~~~

`RayTraceModule`:

~~~ cpp
  // Shader loading and pipeline creation
  VkShaderModule rayTraceModule =
      nvvk::createShaderModule(context, nvh::loadFile("shaders/raytrace.comp.glsl.spv", true, searchPaths));
~~~ cpp highlight
  debugUtil.setObjectName(rayTraceModule, "rayTraceModule");
~~~

and `computePipeline`:

~~~ cpp
  VkPipeline computePipeline;
  NVVK_CHECK(vkCreateComputePipelines(context,                 // Device
                                      VK_NULL_HANDLE,          // Pipeline cache (uses default)
                                      1, &pipelineCreateInfo,  // Compute pipeline create info
                                      nullptr,                 // Allocator (uses default)
                                      &computePipeline));      // Output
~~~ cpp highlight
  debugUtil.setObjectName(computePipeline, "computePipeline");
~~~

The current code shouldn't print out any validation layer warnings or errors, but this should help with debugging Vulkan in the future.



# Instances and Transformation Matrices

In this chapter, we'll add multiple instances of the Cornell box BLAS to the TLAS. Each instance will have a different transformation matrix to make it appear at a different position and rotation within the world. Here's what we'll render at the end of this chapter:

![441 Cornell box BLAS instances. We'll add materials in the next chapter.](images/e9-result.png)

This scene represents 32 x 441 = 14112 triangles, although we only need space in the AS to store 32 triangles for the Cornell box BLAS and data for 441 instances in the TLAS.

!!! Tip
    You can find the reference code at the end of this chapter in the `checkpoints/e9_instances` directory.
    If you run into any problems with the code in this chapter, you can diff your
    files against the reference files to see if there are any differences.

## Transformation Matrices

Transformation matrices allow us to apply combinations of translation, scaling, rotation, and skewing operations -- known as [**affine transformations**](https://en.wikipedia.org/wiki/Affine_transformation#Augmented_matrix) -- to instances. The Vulkan ray tracing API represents a transformation using a 3 x 4 matrix, such as

$$ I = \begin{pmatrix} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \end{pmatrix}. $$

For an instance containing triangle data, the ray tracing implementation acts as if each vertex $v = (x, y, z)$ was transformed using its transformation matrix

$$ M = \begin{pmatrix} M[0][0] & M[0][1] & M[0][2] & M[0][3] \\ M[1][0] & M[1][1] & M[1][2] & M[1][3] \\ M[2][0] & M[2][1] & M[2][2] & M[2][3] \end{pmatrix} $$

as follows, using [matrix multiplication](https://en.wikipedia.org/wiki/Matrix_multiplication):

$$\begin{aligned} M\mathbf{v} &= \begin{pmatrix}M[0][0] & M[0][1] & M[0][2] & M[0][3] \\ M[1][0] & M[1][1] & M[1][2] & M[1][3] \\ M[2][0] & M[2][1] & M[2][2] & M[2][3]\end{pmatrix} \begin{pmatrix}x\\y\\z\\1\end{pmatrix} \\
    &= \begin{pmatrix}M[0][0] & M[0][1] & M[0][2] \\ M[1][0] & M[1][1] & M[1][2] \\ M[2][0] & M[2][1] & M[2][2]\end{pmatrix} \begin{pmatrix}x\\y\\z\end{pmatrix} + \begin{pmatrix}M[0][3]\\M[1][3]\\M[2][3]\end{pmatrix} \\
    &= \begin{pmatrix}M[0][0]x + M[0][1]y + M[0][2]z + M[0][3]\\M[1][0]x + M[1][1]y + M[1][2]z + M[1][3]\\M[2][0]x + M[2][1]y + M[2][2]z + M[2][3]\end{pmatrix}\end{aligned}$$

We'll use the [OpenGL Mathematics (GLM)](https://github.com/g-truc/glm) math
library to compute the transform of an instance. This library uses 4 x 4 matrices
instead of 3 x 4 matrices to represent general transformations, but the last row
will always be `(0, 0, 0, 1)`. When we copy the matrix into
`VkAccelerationStructureInstanceKHR::instance`, which is a
`VkTransformMatrixKHR` (a 3 x 4 matrix), we will discard the last row.

Here are some examples of transformations. For each transformation, we'll show the function(s) used to create it, the 3 x 4 matrix form of the transformation, and a path-traced image of the result.

Remember that the world space coordinate system we're using is a right-handed,
Y-up coordinate system. From the point of view of the camera, the world space +X
axis points right, the +Y axis points up, and the **-Z** axis points into the
screen (since this is the minus Z axis, this means that the positive Z axis
points from the screen towards you). The origin $(0,0,0)$ of the Cornell box and
the world space coordinate system isn't currently in the center of the box; it's
in the middle of the bottom of the floor. The camera is located at
$(-0.001, 1, 6)$. Please note that because we'll need to modify
`raytrace.comp.glsl` a bit, one may not be able to reproduce the shading of all
of these path-traced images until the end of this chapter.

### Including GLM

We'll use [OpenGL Mathematics (GLM)](https://github.com/g-truc/glm), which is
a math library based on objects from GLSL. `glm::mat4` works like GLSL's `mat4`,
`vec4` works like GLSL's `vec4` and so on. It also includes useful extensions
such as built-in matrix translation, rotation, and scaling.

Nvpro_core comes with GLM, so you already have it in your include paths. In
main.cpp, in the list of `#include` directives, add the following two lines:

~~~ cpp highlight
#include <glm/glm.hpp>
#include <glm/gtx/transform.hpp>
~~~ cpp
#include <nvh/fileoperations.hpp>  // For nvh::loadFile
...
~~~

### Identity

The identity matrix doesn't change anything.

~~~ cpp
glm::mat4 transform = glm::identity<glm::mat4>();
~~~

$$ M = \begin{pmatrix} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \end{pmatrix} $$

![Transformation by the identity matrix](images/e9-identity.png)


### Translation

Translation by $(1, 0, 0)$:

~~~ cpp
glm::mat4 transform = glm::translate(glm::vec3(1.0f, 0.0f, 0.0f));
~~~

$$ M = \begin{pmatrix} 1 & 0 & 0 & 1 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \end{pmatrix} $$

![Translation by $(1, 0, 0)$](images/e9-t1,0,0.png)


Translation by $(-1.5, 0, 0)$:

~~~ cpp
glm::mat4 transform = glm::translate(glm::vec3(-1.5f, 0.0f, 0.0f));
~~~

$$ M = \begin{pmatrix} 1 & 0 & 0 & -1.5 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \end{pmatrix} $$

![Translation by $(-1.5, 0, 0)$](images/e9-t-1.5,0,0.png)


Translation by $(0, 0.6, 0)$:

~~~ cpp
glm::mat4 transform = glm::translate(glm::vec3(0.0f, 0.6f, 0.0f));
~~~

$$ M = \begin{pmatrix} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0.6 \\ 0 & 0 & 1 & 0 \end{pmatrix} $$

![Translation by $(0, 0.6, 0)$](images/e9-t0,0.6,0.png)


Translation by $(0, 0, 1)$:

~~~ cpp
glm::mat4 transform = glm::translate(glm::vec3(0.0f, 0.0f, 1.0f));
~~~

$$ M = \begin{pmatrix} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 1 \end{pmatrix} $$

![Translation by $(0, 0, 1)$](images/e9-t0,0,1.png)


### Scaling

!!! Tip
    The origin in world space is in the bottom of the floor of the Cornell box!

Scaling by $(2, 2, 2)$:

~~~ cpp
glm::mat4 transform = glm::scale(glm::vec3(2.0f, 2.0f, 2.0f));
~~~

$$ M = \begin{pmatrix} 2 & 0 & 0 & 0 \\ 0 & 2 & 0 & 0 \\ 0 & 0 & 2 & 0 \end{pmatrix} $$

![Scaling by $(2, 2, 2)$](images/e9-s2,2,2.png)


Scaling by $(0.4, 0.4, 0.4)$:

~~~ cpp
glm::mat4 transform = glm::scale(glm::vec3(0.4f, 0.4f, 0.4f));
~~~

$$ M = \begin{pmatrix} 0.4 & 0 & 0 & 0 \\ 0 & 0.4 & 0 & 0 \\ 0 & 0 & 0.4 & 0 \end{pmatrix} $$

![Scaling by $(0.4, 0.4, 0.4)$](images/e9-s.4,.4,.4.png)

Scaling by $(0.4, 0.9, 0.1)$:

~~~ cpp
glm::mat4 transform = glm::scale(glm::vec3(0.4f, 0.9f, 0.1f));
~~~

$$ M = \begin{pmatrix} 0.4 & 0 & 0 & 0 \\ 0 & 0.9 & 0 & 0 \\ 0 & 0 & 0.1 & 0 \end{pmatrix} $$

![Scaling by $(0.4, 0.9, 0.1)$](images/e9-s.4,.9,.1.png)

!!! Tip
    The matrix formed by the first three columns and rows of the object-to-world matrix must be **invertible**; that is, it must be possible to "undo" the object-to-world matrix. An example of a matrix that isn't invertible is scaling by $(1, 0, 0)$:
    
    $$ M' = \begin{pmatrix} 1 & 0 & 0 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \end{pmatrix} $$

    This is so that the driver can take a 3 x 4 object-to-world matrix and construct the world-to-object matrix that inverts the transformation of the first matrix.

!!! Tip
    Some rendering engines don't support non-uniform scaling like $(0.4, 0.9, 0.1)$ or skewing. This restriction -- only rotation, translation, and uniform scaling -- lets them perform a small optimization: they can transform normals using the object-to-world matrix, instead of spending time computing a matrix proportional to the transpose of the inverse of the object-to-world matrix. More on this in Section [GLSL: Adding Transformation Support].

    This chapter's path tracer supports all invertible matrix operations.

### Rotation

Rotating by $\pi/6$ radians around the +x axis:

~~~ cpp
glm::mat4 transform = glm::rotate(glm::pi<float>() / 6.0f, glm::vec3(1.0f, 0.0f, 0.0f));
~~~

$$ M = \begin{pmatrix} 1 & 0 & 0 & 0 \\ 0 & 0.866025388... & -0.5 & 0 \\ 0 & 0.5 & 0.866025388... & 0 \end{pmatrix} $$

![Rotating around the +x axis](images/e9-r1,0,0.png)


Rotating by $\pi/6$ radians around the -x axis (the inverse of the above rotation):

~~~ cpp
glm::mat4 transform = glm::rotate(glm::pi<float>() / 6.0f, glm::vec3(-1.0f, 0.0f, 0.0f));
~~~

$$ M = \begin{pmatrix} 1 & 0 & 0 & 0 \\ 0 & 0.866025388... & 0.5 & 0 \\ 0 & -0.5 & 0.866025388... & 0 \end{pmatrix} $$

![Rotating around the -x axis](images/e9-r-1,0,0.png)


Rotating by $\pi/6$ radians around the +y axis:

~~~ cpp
glm::mat4 transform = glm::rotate(glm::pi<float>() / 6.0f, glm::vec3(0.0f, 1.0f, 0.0f));
~~~

$$ M = \begin{pmatrix} 0.866025388... & 0 & 0.5 & 0 \\ 0 & 1 & 0 & 0 \\ -0.5 & 0 & 0.866025388... & 0 \end{pmatrix} $$

![Rotating around the +y axis](images/e9-r0,1,0.png)


Rotating by $\pi/6$ radians around the +z axis:

~~~ cpp
glm::mat4 transform = glm::rotate(glm::pi<float>() / 6.0f, glm::vec3(0.0f, 0.0f, 1.0f));
~~~

$$ M = \begin{pmatrix} 0.866025388... & -0.5 & 0 & 0 \\ 0.5 & 0.866025388... & 0 & 0 \\ 0 & 0 & 1 & 0 \end{pmatrix} $$

![Rotating around the +z axis](images/e9-r0,0,1.png)


### Skewing

GLM has functions for skewing, but we need to include a second header for it:

~~~ cpp
#include <glm/gtx/transform2.hpp>
...
glm::mat4 transform = glm::shearY3D(glm::identity<glm::mat4>(), 1.0f, 0.0f);
~~~

$$ M = \begin{pmatrix} 1 & 1 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \end{pmatrix} $$

![A skew transformation.](images/e9-skew.png)


### Combining Operations

Here's how to write a transformation that represents translating the Cornell box down 1 unit, then rotating by $\pi/6$ radians around the +z axis.

~~~ cpp
glm::mat4 transform = glm::translate(glm::vec3(0.0f, -1.0f, 0.0f));
transform           = glm::rotate(glm::pi<float>() / 6.0f, glm::vec3(0.0f, 0.0f, 1.0f)) * transform;
~~~

![Translation followed by rotation.](images/e9-otr.png)

Note the side on which we apply these: when we want to apply another transformation, we apply it on the left-hand side. This is because we transform vectors using the vector on the right-hand side.

For instance, if we wanted to transform a vector `v` by A, then B, then C, we'd write

```
transformedVector = C * (B * (A * v));
```

which is the same as

```
transform         = C * B * A;
transformedVector = transform * v;
```

The order of operations matters here: if we did rotation first, followed by translation, we would get a different transformation:

~~~ cpp
glm::mat4 transform = glm::rotate(glm::pi<float>() / 6.0f, glm::vec3(0.0f, 0.0f, 1.0f));
transform           = glm::translate(glm::vec3(0.0f, -1.0f, 0.0f)) * transform;
~~~

![Rotation followed by translation.](images/e9-ort.png)

We'll start with the more technical side of this chapter -- modifying the GLSL code to support arbitrary invertible transformations -- and then set up the transformations for the instances.

## GLSL: Moving the Camera

Let's start setting up the scene by moving the camera from $(-0.001, 1, 6)$ down and back to $(-0.001, 0, 53)$: in `raytrace.comp.glsl`, change the `const vec3 cameraOrigin` line to

~~~ glsl
  // The camera is located at (-0.001, 0, 53).
  const vec3 cameraOrigin = vec3(-0.001, 0.0, 53.0);
~~~

## GLSL: Adding Transformation Support

`getObjectHitInfo` in `raytrace.comp.glsl` is supposed to return the world space position and normal of the ray query's committed intersection. But currently, it returns the object space position instead! This will be incorrect as soon as we use a matrix other than the identity matrix (although it can produce neat glitches).

Here's how to fix it.

We can get the instance's object-to-world matrix from the ray query using
ray&shy;Query&shy;Get&shy;Intersection&shy;Object&shy;To&shy;World&shy;EXT. Then we write out the transformation of
the intersection point from object space to world space. Replace

~~~ glsl
  // For the main tutorial, object space is the same as world space:
  result.worldPosition = objectPos;
~~~ glsl

with

~~~ glsl
  // Transform from object space to world space:
  const mat4x3 objectToWorld = rayQueryGetIntersectionObjectToWorldEXT(rayQuery, true);
  result.worldPosition       = objectToWorld * vec4(objectPos, 1.0f);
~~~

For normals, we need to use a matrix that is proportional to the **inverse transpose** of the object-to-world matrix, treat the normal as a **direction**, not a point (that is, ignore the translation part of the affine transformation), and normalize at the end. To do this, replace

~~~ glsl
  const vec3 objectNormal = normalize(cross(v1 - v0, v2 - v0));
  // For the main tutorial, object space is the same as world space:
  result.worldNormal = objectNormal;
~~~

with

~~~ glsl
  const vec3 objectNormal = cross(v1 - v0, v2 - v0);
  // Transform normals from object space to world space. These use the transpose of the inverse matrix,
  // because they're directions of normals, not positions:
  const mat4x3 objectToWorldInverse = rayQueryGetIntersectionWorldToObjectEXT(rayQuery, true);
  result.worldNormal                = normalize((objectNormal * objectToWorldInverse).xyz);
~~~

!!! Tip Transposes by Left-Multiplication
    In GLSL, multiplying a matrix by a vector on the **right** represents multiplying
    a matrix by a **column vector**. Suppose `M` is a `mat4x4` (a 4 x 4 matrix) and
    `v` is a `vec4`. Then in GLSL, `M * v` is equal to
    
    $$
    \begin{pmatrix} M[0][0] & M[0][1] & M[0][2] & M[0][3] \\ M[1][0] & M[1][1] & M[1][2] & M[1][3] \\ M[2][0] & M[2][1] & M[2][2] & M[2][3] \\ M[3][0] & M[3][1] & M[3][2] & M[3][3] \end{pmatrix} \begin{pmatrix} v[0] \\ v[1] \\ v[2] \\ v[3] \end{pmatrix}.
    $$
    
    Multiplying a matrix by a vector on the **left** represents multiplying a
    **row vector** by a matrix -- `v * M` is equal to
    
    $$
    \begin{pmatrix} v[0] & v[1] & v[2] & v[3] \end{pmatrix} \begin{pmatrix} M[0][0] & M[0][1] & M[0][2] & M[0][3] \\ M[1][0] & M[1][1] & M[1][2] & M[1][3] \\ M[2][0] & M[2][1] & M[2][2] & M[2][3] \\  M[3][0] & M[3][1] & M[3][2] & M[3][3] \end{pmatrix}.
    $$
    
    Because $$ (A x)^T = x^T A^T $$ for any m x n matrix $A$ and n x 1 column
    vector $x$, where $T$ denotes the transpose, we can use this fact to write
    
    ~~~ glsl
    transpose(objectToWorld) * objectNormal
    ~~~
    
    as
    
    ~~~ glsl
    objectNormal * objectToWorld.
    ~~~
    
    The `.xyz` then removes the translation component.

!!! Tip Why Do Normals Use a Matrix Proportional to the Inverse Transpose?
    Vertex positions are points in space. Triangle normals, on the other hand, are defined by the positions of the three vertices that make up the triangle, so it's not surprising that they would transform differently. This box shows why the inverse transpose works for normals, for readers that have some familiarity with linear algebra!
    
    More formally, the normal vector $n$ of a triangle formed by vertices $v_1$, $v_2$, and $v_3$ must always be perpendicular to the directions of the edges of the triangle, such as $v_2 - v_1$ and $v_3 - v_1$. In other words, we must have

    $$ n \cdot (v_2 - v_1) = 0 \text{ and } n \cdot (v_3 - v_1) = 0 $$
    
    where $\cdot$ is the dot product. (We assume the triangle isn't degenerate, which means we must only consider two edges.)
    
    If the vertices are transformed by a matrix $M$ (we can either consider the left 3 x 3 block of M or use a 4 x 4 version of M with homogeneous $(x, y, z, 1)$ coordinates for points and $(d_x, d_y, d_z, 0)$ coordinates for directions here), then the new normal $n'$ must satisfy
    
    $$ n' \cdot (M v_2 - M v_1) = 0 \text{ and } n' \cdot (M v_3 - M v_1) = 0 $$
    
    or 
    
    $$ n' \cdot (M (v_2 - v_1)) = 0 \text{ and } n' \cdot (M (v_3 - v_1)) = 0 $$
    
    If $c$ is any nonzero constant, then setting
    
    $$ n' = c M^{-T} n $$
    
    works, because of the property of the dot product that for any matrix $A$ and vectors $u, v$, $Au \cdot v = u \cdot A^T v$:
    
    $$ c M^{-T} n \cdot (M (v_2 - v_1)) = c n \cdot (M^{-1} M (v_2 - v_1)) = c n \cdot (v_2 - v_1) = 0 $$
    
    $$ c M^{-T} n \cdot (M (v_3 - v_1)) = c n \cdot (M^{-1} M (v_3 - v_1)) = c n \cdot (v_3 - v_1) = 0 $$
    
    I'm making sure to mention how we can have an arbitrary constant here, because some people prefer using the adjugate of $M$ rather than the inverse transpose of $M$, because the adjugate exists even when $M$ isn't invertible. But since non-invertibility isn't allowed by the ray tracing specification itself, I'm using the inverse transpose here instead.
    
    There are many other ways to interpret this: for instance, one could interpret vertex positions as [contravariant vectors](https://en.wikipedia.org/wiki/Covariance_and_contravariance_of_vectors#Contravariant_transformation) and normals as [covariant vectors](https://en.wikipedia.org/wiki/Covariance_and_contravariance_of_vectors#Covariant_transformation). These also have an interpretation in terms of geometric algebra, which Nathan Reed writes about [here](http://reedbeta.com/blog/normals-inverse-transpose-part-1/).

!!! Tip Other Conventions
    I like to use column vectors for points, because multiplying them by matrices
    on the left reminds me of C++'s function calls.

    ~~~cpp
    transformedVector1 = C * B * A * v;
    transformedVector2 = transformC(transformB(transformA(v)));
    ~~~

    Other applications will sometimes represent points using row vectors, which
    means they'll transpose all their matrices and apply them on the right instead
    of the left.

    Whichever convention your app uses, the most important thing is to be clear
    about it (usually by using comments and descriptive variable names): whether
    points are row vectors or column vectors, whether your coordinate system is
    left-handed or right-handed, and whether the +Y or +Z axis points up. It's
    also important to be careful about whether vectors are points, directions,
    or normals, since they transform differently.

    This is like documenting the inputs to a function: when a function describes
    what values it uses for its inputs, other people (including possibly yourself
    if you re-read your code later) can quickly determine how they should call it.

    It also makes finding bugs easier: if a comment says a shader takes a
    world-to-object transform, but we're giving it an object-to-world transform,
    then we can quickly spot the bug and fix it. (Sometimes it's tempting to try
    to fix a matrix transform bug by flipping signs and permuting axes until the
    test case works, but this tends to be slow since there are 48 possible
    combinations -- 6 XYZ permutations x 2 X signs x 2 Y signs x 2 Z signs -- and
    there's a good chance one's codebase winds up with 2 sign flips instead of 0.)

## Including New Headers

In the rest of this chapter, we'll set up a scene with 441 instances in a grid with random rotations. We'll need the C++ `<random>` header to generate random numbers, so add the following highlighted line to the `#include` block at the top of `main.cpp`:

~~~ cpp
#include <array>
~~~ cpp highlight
#include <random>
~~~ cpp
#define STB_IMAGE_WRITE_IMPLEMENTATION
~~~


## Creating the Scene

To add more instances, we'll add more `VkAccelerationStructureInstanceKHR` structs to the `instances` vector. Each instance will point to the index 0 BLAS in the `blases` vector we passed to `nvvk::RaytracingBuilderKHR::buildBlas`, which is the BLAS for the Cornell box.

We'll start by creating 441 instances (without modifying the transformation matrix quite yet). Replace

~~~ cpp
  // Create an instance pointing to this BLAS, and build it into a TLAS:
  std::vector<VkAccelerationStructureInstanceKHR> instances;
  {
    VkAccelerationStructureInstanceKHR instance{};
    instance.accelerationStructureReference = raytracingBuilder.getBlasDeviceAddress(0);  // The address of the BLAS in `blases` that this instance points to
    // Set the instance transform to the identity matrix:
    instance.transform.matrix[0][0] = instance.transform.matrix[1][1] = instance.transform.matrix[2][2] = 1.0f;
    instance.instanceCustomIndex = 0;  // 24 bits accessible to ray shaders via rayQueryGetIntersectionInstanceCustomIndexEXT
    // Used for a shader offset index, accessible via rayQueryGetIntersectionInstanceShaderBindingTableRecordOffsetEXT
    instance.instanceShaderBindingTableRecordOffset = 0;
    instance.flags = VK_GEOMETRY_INSTANCE_TRIANGLE_FACING_CULL_DISABLE_BIT_KHR;  // How to trace this instance
    instance.mask  = 0xFF;
    instances.push_back(instance);
  }
~~~

with

~~~ cpp highlight
  // Create 441 instances with random rotations pointing to BLAS 0, and build these instances into a TLAS:
~~~ cpp
  std::vector<VkAccelerationStructureInstanceKHR> instances;
~~~ cpp highlight
  std::default_random_engine                      randomEngine;  // The random number generator
  std::uniform_real_distribution<float>           uniformDist(-0.5f, 0.5f);
  for(int x = -10; x <= 10; x++)
  {
    for(int y = -10; y <= 10; y++)
    {
      // Create a 4 x 4 matrix and initialize it to the identity transform
      glm::mat4 transform = glm::identity<glm::mat4>();
      
~~~ cpp
      VkAccelerationStructureInstanceKHR instance{};
      instance.accelerationStructureReference = raytracingBuilder.getBlasDeviceAddress(0);  // The address of the BLAS in `blases` that this instance points to
~~~ cpp highlight
      instance.transform = nvvk::toTransformMatrixKHR(transform);
~~~ cpp
      instance.instanceCustomIndex = 0;  // 24 bits accessible to ray shaders via rayQueryGetIntersectionInstanceCustomIndexEXT
      // Used for a shader offset index, accessible via rayQueryGetIntersectionInstanceShaderBindingTableRecordOffsetEXT
      instance.instanceShaderBindingTableRecordOffset = 0;
      instance.flags = VK_GEOMETRY_INSTANCE_TRIANGLE_FACING_CULL_DISABLE_BIT_KHR;  // How to trace this instance
      instance.mask  = 0xFF;
      instances.push_back(instance);
    }
  }
~~~

Note that in the code above, we also call `nvvk::toTransformMatrixKHR(transform)`; this function takes a 4 x 4 matrix and returns the first 3 rows in a `VkTransformMatrixKHR` struct.

You can now build and run the program -- but it'll run slowly, since we have 21 x 21 = 441 copies of the Cornell box stacked on top of each other! (Overlapping bounding boxes are not good for ray tracing performance, because the traversal engine must consider each instance instead of only a few.)

![441 stacked instances of the Cornell Box.](images/e9-i.png)

Let's modify the transformation matrix to position each instance. First, translate each instance down by 1 unit, so that all the boxes are centered on $(0, 0, 0)$:

~~~ cpp
      glm::mat4 transform = glm::translate(glm::vec3(0.0f, -1.0f, 0.0f));
~~~

![441 stacked instances of the Cornell Box, translated down 1 unit.](images/e9-t.png)

Then randomly rotate each instance between -0.5 and 0.5 radians around the +x axis, by adding a `rotate` operation before the `translate` operation, as described in Section [Combining Operations] above:

~~~ cpp
      glm::mat4 transform = glm::translate(glm::vec3(0.0f, -1.0f, 0.0f));
      transform           = glm::rotate(uniformDist(randomEngine), glm::vec3(1.0f, 0.0f, 0.0f)) * transform;
~~~

![441 instances of the Cornell Box with random +x axis rotations.](images/e9-tr.png)

Then randomly rotate each instance between -0.5 and 0.5 radians around the +y axis:

~~~ cpp
      glm::mat4 transform = glm::translate(glm::vec3(0.0f, -1.0f, 0.0f));
      transform           = glm::rotate(uniformDist(randomEngine), glm::vec3(1.0f, 0.0f, 0.0f)) * transform;
      transform           = glm::rotate(uniformDist(randomEngine), glm::vec3(0.0f, 1.0f, 0.0f)) * transform;
~~~

Now we can see that there's a lot of geometry that we're path tracing!

![441 instances of the Cornell Box with random +x, +y axis rotations.](images/e9-trr.png)

If we translated each instance now, we would get this:

~~~ cpp
      glm::mat4 transform = glm::translate(glm::vec3(0.0f, -1.0f, 0.0f));
      transform           = glm::rotate(uniformDist(randomEngine), glm::vec3(1.0f, 0.0f, 0.0f)) * transform;
      transform           = glm::rotate(uniformDist(randomEngine), glm::vec3(0.0f, 1.0f, 0.0f)) * transform;
      transform           = glm::translate(glm::vec3(float(x), float(y), 0.0f)) * transform;
~~~

![441 instances centered at $(x, y, 0)$, but without scaling beforehand.](images/e9-trrt.png)

Instead of doing that, scale each instance by a factor of 1/2.7. Then translate the instance with index `(x, y)` to be centered at $(x, y, 0)$:

~~~ cpp
      glm::mat4 transform = glm::translate(glm::vec3(0.0f, -1.0f, 0.0f));
      transform           = glm::rotate(uniformDist(randomEngine), glm::vec3(1.0f, 0.0f, 0.0f)) * transform;
      transform           = glm::rotate(uniformDist(randomEngine), glm::vec3(0.0f, 1.0f, 0.0f)) * transform;
      transform           = glm::scale(glm::vec3(1.0f / 2.7f)) * transform;
      transform           = glm::translate(glm::vec3(float(x), float(y), 0.0f)) * transform;
~~~

![441 instances with scaling before translation.](images/e9-trrst.png)

We've now transformed all 441 instances -- and if we look closely, we can see that light can bounce between instances and that instances can shadow each other. For instance, the side of each box now has a complex gradient instead of a uniform color that includes both shadows cast onto them by other boxes, and light bouncing off the sides of other boxes (see the close-up below). The performance should now be back to normal, since the objects' bounding boxes don't overlap as much.

In the next chapter, we'll see how to give each instance a random material!

![Close-up of the scene using `const float fovVerticalSlope = 1.0 / 40.0;`. Note how light can bounce between and shadows can be cast by instances.](images/e9-closeup.png)


# Multiple Materials

In the main tutorial, we mentioned how `instance.instanceCustomIndex` and `instance.instanceShaderBindingTableRecordOffset` can be used to give each instance 24 + 24 = 48 bits of custom data. (For instance, this could be used to index into a buffer that contains much more than 48 bits of data per instances.) We've also been using a diffuse gray material for all of our instances so far -- but in this chapter, we'll see how to add more variety to this scene by adding 9 different materials (expressed as GLSL functions), and using `instance.instanceShaderBindingTableRecordOffset` to select a random material out of these 9 per instance!

Here's what we'll render at the end of this chapter. One might recognize some of these materials from earlier, but we've added a few new ones as well.

![441 instances in a grid; each grid has one of 9 different materials chosen randomly](images/e10-materials.png)

![Close-up of the scene showing all 9 materials, generated using `fovVerticalSlope = 1.0 / 24.0`. Observe how different materials overlap, reflect light off, and shadow each other.](images/e10-closeup.png)


## Random Material IDs

To start, let's give each instance a random `instanceShaderBindingTableRecordOffset` between 0 and 8, holding the index of their material.

!!! Tip
    You can find the reference code at the end of this chapter in the `checkpoints/e10_materials` directory.
    If you run into any problems with the code in this chapter, you can diff your
    files against the reference files to see if there are any differences.

First, define a distribution that randomly selects integers from 0 to 8 inclusive by adding the highlighted line in `main.cpp`:

~~~ cpp
  std::default_random_engine                        randomEngine;  // The random number generator
  std::uniform_real_distribution<float>             uniformDist(-0.5f, 0.5f);
~~~ cpp highlight
  std::uniform_int_distribution<int>                uniformIntDist(0, 8);
~~~

Then set `instance.instanceShaderBindingTableRecordOffset` to a random integer from 0 to 8 inclusive for each instance by replacing

~~~ cpp
      // Used for a shader offset index, accessible via rayQueryGetIntersectionInstanceShaderBindingTableRecordOffsetEXT
      instance.instanceShaderBindingTableRecordOffset = 0;
~~~

with

~~~ cpp
      // Used for a shader offset index, accessible via rayQueryGetIntersectionInstanceShaderBindingTableRecordOffsetEXT
      instance.instanceShaderBindingTableRecordOffset = uniformIntDist(randomEngine);
~~~

That's all the C++ code for this chapter!


## A Common Header File for Materials and Material Shaders

We'll now create a file called `shaderCommon.h` that will include common code for different materials (this naming will make more sense in the next chapter, when this file contains common code for different material *shaders*) and define 9 functions representing material models.

The idea here is that we're emulating an application where artists and programmers can write materials for objects, and where a scene might have hundreds or thousands of different materials (which is likely for many games and applications as of this writing).
For instance, many tools like Unity and Blender let users define materials by joining nodes into node graphs. In Blender, these node graphs are then compiled or interpreted to or from GLSL, OptiX, CUDA, and a CPU-based system. For this tutorial, we've simplified this a bit by including all 9 functions in a single GLSL file, although we'll show cleaner and faster ways of handling multiple material models in the next chapter.

Each material function will take a `rayQueryEXT` object and an `inout uint` for the random number generator state (so that it can choose random reflections), and will return the reflective color of the surface, a new ray origin, and a new ray direction.

At some point, every material function will call `getObjectHitInfo` to get the world space position and normal of the intersection. We'll also modify `getObjectHitInfo` to return the object space position, which some materials will use.

Every material function will also call a new function called `offsetPositionAlongNormal` to move the ray origin off the intersected triangle to avoid self-intersection. In previous chapters, we used an implementation like this:

~~~ cpp
vec3 offsetPositionAlongNormal(vec3 worldPosition, vec3 normal)
{
  return worldPosition + 0.0001 * normal;
}
~~~

However, we now include a more accurate implementation in this file, from Carsten Wächter and Nikolaus Binder's paper *A Fast and Robust Method for Avoiding Self-Intersection* from [Ray Tracing Gems](http://www.realtimerendering.com/raytracinggems/).

Because this new file is a bit long, we recommend copying and pasting it instead of typing it in. We've included comments along the way for interested readers to learn more about how these material models work.

Create a new file called `shaderCommon.h` inside the `shaders` subdirectory, and fill it with the following:

~~~ glsl
// Common file defining all the functions used for materials.
// In the next chapter, this file will be shared across closest-hit shaders.
#ifndef VK_MINI_PATH_TRACER_SHADER_COMMON_H
#define VK_MINI_PATH_TRACER_SHADER_COMMON_H

// Info retrieved from a rayQueryEXT by getObjectHitInfo.
struct HitInfo
{
  vec3 objectPosition;  // The intersection position in object-space.
  vec3 worldPosition;   // The intersection position in world-space.
  vec3 worldNormal;     // The double-sided triangle normal in world-space.
};

HitInfo getObjectHitInfo(rayQueryEXT rayQuery)
{
  HitInfo result;
  // Get the ID of the triangle
  const int primitiveID = rayQueryGetIntersectionPrimitiveIndexEXT(rayQuery, true);

  // Get the indices of the vertices of the triangle
  const uint i0 = indices[3 * primitiveID + 0];
  const uint i1 = indices[3 * primitiveID + 1];
  const uint i2 = indices[3 * primitiveID + 2];

  // Get the vertices of the triangle
  const vec3 v0 = vertices[i0];
  const vec3 v1 = vertices[i1];
  const vec3 v2 = vertices[i2];


  // Get the barycentric coordinates of the intersection
  vec3 barycentrics = vec3(0.0, rayQueryGetIntersectionBarycentricsEXT(rayQuery, true));
  barycentrics.x    = 1.0 - barycentrics.y - barycentrics.z;

  // Compute the coordinates of the intersection
  result.objectPosition = v0 * barycentrics.x + v1 * barycentrics.y + v2 * barycentrics.z;
  // Transform from object space to world space:
  const mat4x3 objectToWorld = rayQueryGetIntersectionObjectToWorldEXT(rayQuery, true);
  result.worldPosition       = objectToWorld * vec4(result.objectPosition, 1.0f);


  // Compute the normal of the triangle in object space, using the right-hand rule:
  //    v2      .
  //    |\      .
  //    | \     .
  //    |/ \    .
  //    /   \   .
  //   /|    \  .
  //  L v0---v1 .
  // n
  const vec3 objectNormal = cross(v1 - v0, v2 - v0);
  // Transform normals from object space to world space. These use the transpose of the inverse matrix,
  // because they're directions of normals, not positions:
  const mat4x3 objectToWorldInverse = rayQueryGetIntersectionWorldToObjectEXT(rayQuery, true);
  result.worldNormal                = normalize((objectNormal * objectToWorldInverse).xyz);

  // Flip the normal so it points against the ray direction:
  const vec3 rayDirection = rayQueryGetWorldRayDirectionEXT(rayQuery);
  result.worldNormal      = faceforward(result.worldNormal, rayDirection, result.worldNormal);

  return result;
}

// offsetPositionAlongNormal shifts a point on a triangle surface so that a
// ray bouncing off the surface with tMin = 0.0 is no longer treated as
// intersecting the surface it originated from.
//
// Here's the old implementation of it we used in earlier chapters:
// vec3 offsetPositionAlongNormal(vec3 worldPosition, vec3 normal)
// {
//   return worldPosition + 0.0001 * normal;
// }
//
// However, this code uses an improved technique by Carsten Wächter and
// Nikolaus Binder from "A Fast and Robust Method for Avoiding
// Self-Intersection" from Ray Tracing Gems (version 1.7, 2020).
// The normal can be negated if one wants the ray to pass through
// the surface instead.
vec3 offsetPositionAlongNormal(vec3 worldPosition, vec3 normal)
{
  // Convert the normal to an integer offset.
  const float int_scale = 256.0f;
  const ivec3 of_i      = ivec3(int_scale * normal);

  // Offset each component of worldPosition using its binary representation.
  // Handle the sign bits correctly.
  const vec3 p_i = vec3(  //
      intBitsToFloat(floatBitsToInt(worldPosition.x) + ((worldPosition.x < 0) ? -of_i.x : of_i.x)),
      intBitsToFloat(floatBitsToInt(worldPosition.y) + ((worldPosition.y < 0) ? -of_i.y : of_i.y)),
      intBitsToFloat(floatBitsToInt(worldPosition.z) + ((worldPosition.z < 0) ? -of_i.z : of_i.z)));

  // Use a floating-point offset instead for points near (0,0,0), the origin.
  const float origin     = 1.0f / 32.0f;
  const float floatScale = 1.0f / 65536.0f;
  return vec3(  //
      abs(worldPosition.x) < origin ? worldPosition.x + floatScale * normal.x : p_i.x,
      abs(worldPosition.y) < origin ? worldPosition.y + floatScale * normal.y : p_i.y,
      abs(worldPosition.z) < origin ? worldPosition.z + floatScale * normal.z : p_i.z);
}

// Steps the RNG and returns a floating-point value between 0 and 1 inclusive.
float stepAndOutputRNGFloat(inout uint rngState)
{
  // Condensed version of pcg_output_rxs_m_xs_32_32, with simple conversion to floating-point [0,1].
  rngState  = rngState * 747796405 + 1;
  uint word = ((rngState >> ((rngState >> 28) + 4)) ^ rngState) * 277803737;
  word      = (word >> 22) ^ word;
  return float(word) / 4294967295.0f;
}

const float k_pi = 3.14159265;

// The values returned by a material function to the main path tracing routine.
struct ReturnedInfo
{
  vec3 color;         // The reflectivity of the surface.
  vec3 rayOrigin;     // The new ray origin in world-space.
  vec3 rayDirection;  // The new ray direction in world-space.
};

// Returns a random diffuse (Lambertian) reflection for a surface with the
// given normal, using the given random number generator state. This is
// cosine-weighted, so directions closer to the normal are more likely to
// be chosen.
vec3 diffuseReflection(vec3 normal, inout uint rngState)
{
  // For a random diffuse bounce direction, we follow the approach of
  // Ray Tracing in One Weekend, and generate a random point on a sphere
  // of radius 1 centered at the normal. This uses the random_unit_vector
  // function from chapter 8.5:
  const float theta     = 2.0 * k_pi * stepAndOutputRNGFloat(rngState);  // Random in [0, 2pi]
  const float u         = 2.0 * stepAndOutputRNGFloat(rngState) - 1.0;   // Random in [-1, 1]
  const float r         = sqrt(1.0 - u * u);
  const vec3  direction = normal + vec3(r * cos(theta), r * sin(theta), u);

  // Then normalize the ray direction:
  return normalize(direction);
}

// Diffuse reflection off a 70% reflective surface (what we've used for most
// of this tutorial)
ReturnedInfo material0(rayQueryEXT rayQuery, inout uint rngState)
{
  HitInfo hitInfo = getObjectHitInfo(rayQuery);

  ReturnedInfo result;
  result.color        = vec3(0.7);
  result.rayOrigin    = offsetPositionAlongNormal(hitInfo.worldPosition, hitInfo.worldNormal);
  result.rayDirection = diffuseReflection(hitInfo.worldNormal, rngState);

  return result;
}

// A mirror-reflective material that absorbs 30% of incoming light.
ReturnedInfo material1(rayQueryEXT rayQuery, inout uint rngState)
{
  HitInfo hitInfo = getObjectHitInfo(rayQuery);

  ReturnedInfo result;
  result.color        = vec3(0.7);
  result.rayOrigin    = offsetPositionAlongNormal(hitInfo.worldPosition, hitInfo.worldNormal);
  result.rayDirection = reflect(rayQueryGetWorldRayDirectionEXT(rayQuery), hitInfo.worldNormal);

  return result;
}

// A diffuse surface with faces colored according to their world-space normal.
ReturnedInfo material2(rayQueryEXT rayQuery, inout uint rngState)
{
  HitInfo hitInfo = getObjectHitInfo(rayQuery);

  ReturnedInfo result;
  result.color        = vec3(0.5) + 0.5 * hitInfo.worldNormal;
  result.rayOrigin    = offsetPositionAlongNormal(hitInfo.worldPosition, hitInfo.worldNormal);
  result.rayDirection = diffuseReflection(hitInfo.worldNormal, rngState);

  return result;
}

// A linear blend of 20% of a mirror-reflective material and 80% of a perfectly
// diffuse material.
ReturnedInfo material3(rayQueryEXT rayQuery, inout uint rngState)
{
  HitInfo hitInfo = getObjectHitInfo(rayQuery);

  ReturnedInfo result;
  result.color     = vec3(0.7);
  result.rayOrigin = offsetPositionAlongNormal(hitInfo.worldPosition, hitInfo.worldNormal);
  if(stepAndOutputRNGFloat(rngState) < 0.2)
  {
    result.rayDirection = reflect(rayQueryGetWorldRayDirectionEXT(rayQuery), hitInfo.worldNormal);
  }
  else
  {
    result.rayDirection = diffuseReflection(hitInfo.worldNormal, rngState);
  }

  return result;
}

// A material where 50% of incoming rays pass through the surface (treating it
// as transparent), and the other 50% bounce off using diffuse reflection.
ReturnedInfo material4(rayQueryEXT rayQuery, inout uint rngState)
{
  HitInfo hitInfo = getObjectHitInfo(rayQuery);

  ReturnedInfo result;
  result.color = vec3(0.7);
  if(stepAndOutputRNGFloat(rngState) < 0.5)
  {
    result.rayOrigin    = offsetPositionAlongNormal(hitInfo.worldPosition, hitInfo.worldNormal);
    result.rayDirection = diffuseReflection(hitInfo.worldNormal, rngState);
  }
  else
  {
    result.rayOrigin    = offsetPositionAlongNormal(hitInfo.worldPosition, -hitInfo.worldNormal);
    result.rayDirection = rayQueryGetWorldRayDirectionEXT(rayQuery);
  }

  return result;
}

// A material with diffuse reflection that is transparent whenever
// (x + y + z) % 0.5 < 0.25 in object-space coordinates.
ReturnedInfo material5(rayQueryEXT rayQuery, inout uint rngState)
{
  HitInfo hitInfo = getObjectHitInfo(rayQuery);

  ReturnedInfo result;
  if(mod(dot(hitInfo.objectPosition, vec3(1, 1, 1)), 0.5) >= 0.25)
  {
    result.color        = vec3(0.7);
    result.rayOrigin    = offsetPositionAlongNormal(hitInfo.worldPosition, hitInfo.worldNormal);
    result.rayDirection = diffuseReflection(hitInfo.worldNormal, rngState);
  }
  else
  {
    result.color        = vec3(1.0);
    result.rayOrigin    = offsetPositionAlongNormal(hitInfo.worldPosition, -hitInfo.worldNormal);
    result.rayDirection = rayQueryGetWorldRayDirectionEXT(rayQuery);
  }

  return result;
}

// A mirror material that uses normal mapping: we perturb the geometric
// (triangle) normal to get a shading normal that varies over the surface, and
// then use the shading normal to get reflections. This is often used with a
// texture called a normal map in real-time graphics, because it can make it
// look like an object has details that aren't there in the geometry. In this
// function, we perturb the normal without textures using a mathematical
// function instead.
// There's a lot of depth (no pun intended) in normal mapping; two things to
// note in this example are:
// - It's not well-defined what happens when normal mapping produces a
// direction that goes through the surface. In this function we mirror it so
// that it doesn't go through the surface; in a different path tracer, we might
// reject this ray by setting its sample weight to 0, or do something more
// sophisticated.
// - When a BRDF (bidirectional reflectance distribution function; describes
// how much light from direction A bounces off a material in direction B) uses
// a shading normal instead of a geometric normal for shading, the BRDF has to
// be corrected in order to make the math physically correct and to avoid
// errors in bidirectional path tracers. This function ignores that (we don't
// describe BRDFs or sample weights in this tutorial!), but the authoritative
// source for how to do this is chapters 5-7 of Eric Veach's Ph.D. thesis,
// "Robust Monte Carlo Methods for Light Transport Simulation", available for
// free online.
ReturnedInfo material6(rayQueryEXT rayQuery, inout uint rngState)
{
  HitInfo hitInfo = getObjectHitInfo(rayQuery);

  ReturnedInfo result;
  result.color     = vec3(0.7);
  result.rayOrigin = offsetPositionAlongNormal(hitInfo.worldPosition, hitInfo.worldNormal);

  // Perturb the normal:
  const float scaleFactor        = 80.0;
  const vec3  perturbationAmount = 0.03
                                  * vec3(sin(scaleFactor * hitInfo.worldPosition.x),  //
                                         sin(scaleFactor * hitInfo.worldPosition.y),  //
                                         sin(scaleFactor * hitInfo.worldPosition.z));
  const vec3 shadingNormal = normalize(hitInfo.worldNormal + perturbationAmount);
  if(stepAndOutputRNGFloat(rngState) < 0.4)
  {
    result.rayDirection = reflect(rayQueryGetWorldRayDirectionEXT(rayQuery), shadingNormal);
  }
  else
  {
    result.rayDirection = diffuseReflection(shadingNormal, rngState);
  }
  // If the ray now points into the surface, reflect it across:
  if(dot(result.rayDirection, hitInfo.worldNormal) <= 0.0)
  {
    result.rayDirection = reflect(result.rayDirection, hitInfo.worldNormal);
  }

  return result;
}

// A diffuse material where the color of each triangle is determined by its
// primitive ID (the index of the triangle in the BLAS)
ReturnedInfo material7(rayQueryEXT rayQuery, inout uint rngState)
{
  HitInfo hitInfo = getObjectHitInfo(rayQuery);

  ReturnedInfo result;
  const int    primitiveID = rayQueryGetIntersectionPrimitiveIndexEXT(rayQuery, true);
  result.color        = clamp(vec3(primitiveID / 36.0, primitiveID / 9.0, primitiveID / 18.0), vec3(0.0), vec3(1.0));
  result.rayOrigin    = offsetPositionAlongNormal(hitInfo.worldPosition, hitInfo.worldNormal);
  result.rayDirection = diffuseReflection(hitInfo.worldNormal, rngState);

  return result;
}

// A diffuse material with transparent cutouts arranged in slices of spheres.
ReturnedInfo material8(rayQueryEXT rayQuery, inout uint rngState)
{
  HitInfo hitInfo = getObjectHitInfo(rayQuery);

  ReturnedInfo result;
  if(mod(length(hitInfo.objectPosition), 0.2) >= 0.05)
  {
    result.color        = vec3(0.7);
    result.rayOrigin    = offsetPositionAlongNormal(hitInfo.worldPosition, hitInfo.worldNormal);
    result.rayDirection = diffuseReflection(hitInfo.worldNormal, rngState);
  }
  else
  {
    result.color        = vec3(1.0);
    result.rayOrigin    = offsetPositionAlongNormal(hitInfo.worldPosition, -hitInfo.worldNormal);
    result.rayDirection = rayQueryGetWorldRayDirectionEXT(rayQuery);
  }

  return result;
}

#endif  // #ifndef VK_MINI_PATH_TRACER_SHADER_COMMON_H
~~~


## Modifying `raytrace.comp.glsl`

Let's now modify the path tracing code in `raytrace.comp.glsl` to update its rays by getting the custom instance ID from an intersection, `switch`ing over this instance ID, and calling the corresponding material function.

First, add the following highlighted lines **after** the definition of the push constants. Because `#include` directives act like text substitution, this makes it so that the functions in `shaderCommon.h` can access the `vertices` and `indices` arrays.

~~~ glsl
layout(push_constant) uniform PushConsts
{
  PushConstants pushConstants;
};
~~~ glsl highlight

#include "shaderCommon.h"
~~~

Then remove `stepAndOutputRNGFloat`, `k_pi`, `HitInfo`, and `getObjectHitInfo` from `raytrace.comp.glsl`, since we moved these to `shaderCommon.h`.

Finally, the code describing what to do if the ray query intersected a triangle (that is, if `rayQueryGetIntersectionTypeEXT(rayQuery, true) == gl_RayQueryCommittedIntersectionTriangleEXT`) currently looks like this:

~~~ glsl
        // Ray hit a triangle
        HitInfo hitInfo = getObjectHitInfo(rayQuery);

        // Apply color absorption
        accumulatedRayColor *= hitInfo.color;

        // Flip the normal so it points against the ray direction:
        hitInfo.worldNormal = faceforward(hitInfo.worldNormal, rayDirection, hitInfo.worldNormal);

        // Start a new ray at the hit position, but offset it slightly along the normal:
        rayOrigin = hitInfo.worldPosition + 0.0001 * hitInfo.worldNormal;

        // For a random diffuse bounce direction, we follow the approach of
        // Ray Tracing in One Weekend, and generate a random point on a sphere
        // of radius 1 centered at the normal. This uses the random_unit_vector
        // function from chapter 8.5:
        const float theta = 2.0 * k_pi * stepAndOutputRNGFloat(rngState);  // Random in [0, 2pi]
        const float u     = 2.0 * stepAndOutputRNGFloat(rngState) - 1.0;   // Random in [-1, 1]
        const float r     = sqrt(1.0 - u * u);
        rayDirection      = hitInfo.worldNormal + vec3(r * cos(theta), r * sin(theta), u);
        // Then normalize the ray direction:
        rayDirection = normalize(rayDirection);
~~~

Replace this code with the following, which looks up the instance ID of the intersected instance, then switches over calls to different materials. Once it receives the `ReturnedInfo` object, from the material, it updates the accumulated ray color, and starts a new ray at the returned origin and direction:

~~~ glsl
        // Get the ID of the shader:
        const int sbtOffset = int(rayQueryGetIntersectionInstanceShaderBindingTableRecordOffsetEXT(rayQuery, true));

        // Get information about the absorption, new ray origin, and new ray color:
        ReturnedInfo returnedInfo;
        switch(sbtOffset)
        {
          case 0:
            returnedInfo = material0(rayQuery, rngState);
            break;
          case 1:
            returnedInfo = material1(rayQuery, rngState);
            break;
          case 2:
            returnedInfo = material2(rayQuery, rngState);
            break;
          case 3:
            returnedInfo = material3(rayQuery, rngState);
            break;
          case 4:
            returnedInfo = material4(rayQuery, rngState);
            break;
          case 5:
            returnedInfo = material5(rayQuery, rngState);
            break;
          case 6:
            returnedInfo = material6(rayQuery, rngState);
            break;
          case 7:
            returnedInfo = material7(rayQuery, rngState);
            break;
          default:
            returnedInfo = material8(rayQuery, rngState);
            break;
        }

        // Apply color absorption
        accumulatedRayColor *= returnedInfo.color;

        // Start a new segment
        rayOrigin    = returnedInfo.rayOrigin;
        rayDirection = returnedInfo.rayDirection;
~~~

Some readers may be wondering what the performance of this `switch` statement is like, how this scales as the number of shaders increases, and how extensible this is as the number of shaders increases. The answer is that this is not the preferred way to handle many materials. Here's why:

- **Shader divergence**: As mentioned above, all invocations in a **warp** (a group of 32 threads) of a work group must run over the same instructions (and an execution mask keeps track of which threads are active, making branches possible). This is a SIMT architecture: the same instruction runs in parallel over 32 threads. In other words, if two different invocations or **threads** in a warp go down different branches in a statement with branching, such as an `if` or `switch` statement, or a `for` or `while` loop, warp execution must traverse both branches!
 - In this scene, the rays in a warp at any given time will often intersect triangles of instances with different materials. For instance, if 16 threads have `sbtOffset == 2`, 9 threads have `sbtOffset == 4`, 6 threads have `sbtOffset == 7`, and 1 thread has `sbtOffset == 8`, then the warp will have to evaluate all of materials 2, 4, 7, and 8. This can get worse as the number of materials increases, the scene complexity increases, or as materials get more complex and require more instructions to evaluate.

- **Instruction cache inefficiency**: With hundreds of shaders, it's likely that
  this compiled compute shader will have tens of thousands of instructions.
  When code execution reaches the large switch statement, it'll essentially jump
  to a random place within those tens of thousands of instructions
  (possibly multiple times with shader divergence as mentioned above).
  Normally, instruction caches (which both CPUs and GPUs have) speed up
  execution by fetching instructions in advance -- but in this case, we'll get a
  cache miss, which will be slow.

- **Recompilation time**: If a single material is added to the scene, we'll have to recompile the entire `raytrace.comp.glsl` file. If this referenced thousands of complex materials, this could take a long time.

Luckily, there's a better way to handle this: in the next chapter, we'll introduce **ray tracing pipelines**, a type of pipeline that greatly extends the capabilities of shaders, makes scenarios like this faster, and can be used for things other than ray tracing!

You should now be able to build and run the application and see the 441 instances have 9 different materials:

![](images/e10-materials.png)

As one last image for this chapter, here's what this scene looks like with antialiasing turned off. Because of the higher level of detail, it's now much easier to see how antialiasing helps with visual quality in this scene!

![This scene rendered without antialiasing.](images/e10-alias.png)



# Ray Tracing Pipelines

Up to this point in the tutorial, we've been using compute pipelines and
compute shaders. Ray tracing pipelines are another type of `VkPipeline` that
greatly extends the capabilities of shaders in Vulkan, even if they don't use
ray tracing (e.g. when mimicking a compute shader). With ray tracing pipelines,
shaders can call other shaders like calling functions, instances can have their
own shaders, shaders can be structured and called in a way that helps with some
of the issues with last chapter's approach, and it's no longer necessary to
worry about work group sizes. The technology underlying ray tracing pipelines
has been in NVIDIA GPUs for a few years now, starting with DirectX Raytracing
(2018) and the NVIDIA RTX series of GPUs. They were introduced to Vulkan with
`VK_NV_ray_tracing`, and are now available as part of the cross-platform
`VK_KHR_ray_tracing_pipeline` Vulkan extension.

In this tutorial, we'll introduce ray tracing pipelines in a few steps. First, we'll describe some of the new features in ray tracing pipelines, to give a high-level overview before we start modifying the code. Since ray generation shaders are like compute shaders, we'll then modify the application to use a ray tracing pipeline that works like a compute pipeline. We'll then add a closest-hit shader to represent a material, and render a scene where all instances map to that shader group. Finally, we'll have each instance map to a random shader group within a table of shader groups.

## Overview

There are a few new aspects of ray tracing pipelines that we haven't encountered until now. Here are a few of them.

### Callable Shaders

When we introduced shaders, we used the analogy that shaders are like functions that a GPU can run. This is because GPUs can call shaders in different ways: a GPU acts as if it calls

* a compute shader for each invocation in each work group;
* a ray generation shader for each invocation in the dispatch;
* a vertex shader for each vertex in a draw call;

and so on. But what if shaders could call other shaders, like how functions can call functions using function pointers in C++? This is now possible using Vulkan ray tracing pipelines. This involves storing groups of shader handles in tables (arrays) like this (that we'll describe shortly):

![Four shader binding tables stored consecutively. The last contains three callable shader handles. From the [NVIDIA Vulkan Ray Tracing Callable Shaders - Tutorial](https://github.com/nvpro-samples/vk_raytracing_tutorial_KHR/tree/master/ray_tracing_callable).](images/e11-sbt.png)

We can pass pointers to these tables to `vkCmdTraceRaysKHR`, and then GLSL shaders can call **callable shaders** like this:

~~~ cpp
  executeCallableEXT(pushC.lightType, 0);
~~~

We won't directly use callable shaders in this tutorial (please see the [Callable Shaders - Tutorial](https://github.com/nvpro-samples/vk_raytracing_tutorial_KHR/tree/master/ray_tracing_callable) page of the NVIDIA Vulkan Ray Tracing Tutorial for a tutorial), but these have many uses. Imagine sharing common code between shaders using callable shaders, or using callable shaders like lambda functions (e.g. for lighting in the NVIDIA Vulkan Ray Tracing Tutorial). Callable shaders can even be called recursively, up to a limit (a bit like the limit on stack size for CPU code)!

The idea that shaders can call other shaders is core to the idea of the ray tracing pipeline.

### Passing Data Between Shaders

When we call a function in C++, we can pass data to and get data back from a function. For instance, we can use a `int& data` argument to share an `int` with a function -- whether the caller's written that int, or whether the callee will write that int.

In GLSL, we can pass data between shaders using **ray payloads** (when calling a ray tracing shader) and **callable data** (when calling a callable shader). For instance, shader A in `raytrace.rgen.glsl` will have some GLSL code like this:

~~~ glsl
struct PassableInfo
{
  vec3 color;         // The reflectivity of the surface.
  vec3 rayOrigin;     // The new ray origin in world-space.
  vec3 rayDirection;  // The new ray direction in world-space.
  uint rngState;      // State of the random number generator.
  bool rayHitSky;     // True if the ray hit the sky.
};
~~~ glsl highlight

layout(location = 0) rayPayloadEXT PassableInfo pld;
~~~

Shader A will trace a ray into the scene, telling called shaders to use the `rayPayloadEXT` at location 0:

~~~ glsl
traceRayEXT(..., // Other things we'll cover later
            0);  // Location of payload
~~~

This might call entry point `main` of shader B, which will have some GLSL code like this using `rayPayloadInEXT`:

~~~ glsl
struct PassableInfo
{
  vec3 color;         // The reflectivity of the surface.
  vec3 rayOrigin;     // The new ray origin in world-space.
  vec3 rayDirection;  // The new ray direction in world-space.
  uint rngState;      // State of the random number generator.
  bool rayHitSky;     // True if the ray hit the sky.
};
~~~ glsl highlight

layout(location = 0) rayPayloadInEXT PassableInfo pld;

void main()
{
  pld.color = vec3(0.7);
}
~~~

When control passes back to shader A, shader A will see that its `pld.color` is set to `vec3(0.7)`.

In this chapter, we'll use payloads to have each shader implementing a material pass back the new color, ray origin, ray direction, RNG state, and whether the ray hit the sky.

<!-- Custom admonition to handle diagrams -->
<div class ="admonition tip">
<div class="admonition-title">More About Payloads and Callable Data</div>
    There are a couple of things to note here (thanks to David Akeley, Eric Werness, Christoph Kubisch, and Daniel Koch for researching and clarifying this).
    
    Together, all the blocks that we've been marking with layout qualifiers like
    
    ~~~ glsl
    layout(binding = BINDING_INDICES, set = 0, scalar) buffer Indices
    {
      uint indices[];
    };
    ~~~
    
    and
    
    ~~~ glsl
    layout(location = 0) rayPayloadInEXT PassableInfo pld;
    ~~~
    
    form the **linkage** of the shader: they say how the shader "links" with other things, sharing data with the application, API, and with other shaders.
    
    Each shader can only have one `rayPayloadInEXT` variable (just as like a function in C++ can only have one set of arguments, counting overloaded functions as different functions). This means that the input location of a called shader is unambiguous.
    
    The location of the `rayPayloadInEXT` variable in the called shader does **not** have to match the location of the `rayPayloadEXT` variable selected in the shader that called it. The `payload` argument in `traceRayEXT` only selects which payload to use. So, for instance, we could have had
    
    ~~~ glsl
    layout(location = 3) rayPayloadInEXT PassableInfo pld;
    ~~~
    
    and that would have been fine since the location of the `rayPayloadInEXT` variable is unambiguous (since there's only one).
    
    However, the type of the payload on each side must match; otherwise, you'll get undefined behavior. We'll use a common file to reduce code duplication.
    
    Each location can only be used by one variable, so this isn't valid GLSL:
    
    ~~~ glsl
    layout(location = 0) rayPayloadEXT PassableInfo pld;
    layout(location = 0) rayPayloadInEXT PassableInfo pld;
    ~~~
    
    The `payload` argument in `traceRayEXT` can select either a `rayPayloadEXT` variable, or a `rayPayloadInEXT` variable! If we have a call chain of shaders like this:
    
    ******************************************************************
    *
    * .-------------.     .-----------------.     .-----------------.
    * |             |     | in location = 0 |     | in location = 1 |
    * | Shader A    | --> | Shader B        | --> | Shader C        |
    * | payload = 0 |     | payload = 0     |     |                 |
    * '-------------'     '-----------------'     '-----------------'
    *
    ******************************************************************
    
    then all three shaders act as if they share the same payload. (Here, Shader B selected a `rayPayloadInEXT` variable when choosing `payload = 0`.)
    
    This also all holds for callable shaders and callable data.
</div>

### Ray Traversal

In the previous chapter, each instance used one of 9 materials. In this chapter, each instance will use one of 9 **closest-hit** shaders. Here's how that works.

![Diagram of ray tracing pipeline execution. From the [Ray Tracing Pipeline](https://nvpro-samples.github.io/vk_raytracing_tutorial_KHR/#raytracingpipeline) chapter of the NVIDIA Vulkan Ray Tracing Tutorial.](images/e11-ShaderPipeline.svg)

Like the current compute shader, a **ray generation** shader starts by launching ray tracing calls, using `traceRayEXT`. The GPU then traverses the acceleration structure to find where the ray might intersect the scene. Each instance might have data telling the traversal engine to look at a different group of shaders.

For every potential intersection with a triangle or procedural instance, traversal does the following:

* If the instance is procedural (i.e. is a bounding box instead of a triangle mesh), then traversal calls an **intersection shader**. This can tell traversal that there were intersections at different t-values (see [this tutorial](https://github.com/nvpro-samples/vk_raytracing_tutorial_KHR/tree/master/ray_tracing_intersection)) and generate more potential intersections for the any hit shader to process.

* If the shader group has an any hit shader, then traversal calls the **any hit shader**. (This is kind of like the code we could write inside the `rayQueryProceedEXT` loop [here](index.html#fourusesofintersectiondata/countingintersectionswithrayqueryproceedext)). Any hit shaders can tell traversal to conditionally ignore potential intersections, changing the closest intersection (and can also perform arbitrary other code). This makes them sometimes used for transparency and cutout materials. Note that these can be slow, because these must be called for every intersection (which isn't the case for closest-hit shaders)! See the corresponding NVIDIA Vulkan Ray Tracing tutorial chapter [here](https://github.com/nvpro-samples/vk_raytracing_tutorial_KHR/tree/master/ray_tracing_anyhit).

If the ray didn't intersect anything (which is usually very fast), traversal calls the **miss shader**. Otherwise, traversal calls the **closest hit shader**.

As described above, traversal can pass a payload to each of these shaders. In this chapter, we'll use one ray generation shader for the path tracer loop, one miss shader for the sky (which will set `rayHitSky` to `true`), and 9 closest hit shaders to implement materials.

In hardware, RT Cores implement ray traversal using a model like the one in the bottom half of this figure:

![A block diagram of ray traversal using RT Cores. From the [NVIDIA Turing GPU Architecture whitepaper](https://www.nvidia.com/content/dam/en-zz/Solutions/design-visualization/technologies/turing-architecture/NVIDIA-Turing-Architecture-Whitepaper.pdf).](images/e11-rt-core.png)

### Shader Binding Tables

A [shader binding table](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/html/chap38.html#shader-binding-table) (SBT) is an array in
memory, stored inside a `VkBuffer`.
The stride between elements (called **records**) in each SBT can be chosen by
the application within limits (see Section [Calculating SBT Stride])
The first part of each record is a **shader group handle** referencing up
to three shaders.
The rest of the data up to the next record is a **shader record**, which can be
any data you want, accessible via a `shaderRecordEXT` variable in GLSL.

!!! Tip Naming Differences between Vulkan and OptiX
    In OptiX, a shader binding table element is called a record, and consists of
    a **header** and a **payload**. In Vulkan, SBT elements are also called
    records, but each consist of a **shader group handle** and a
    **shader record** (accessible in SPIR-V via a `ShaderRecordBufferKHR` block).

Although shader binding tables are known for sometimes being complex, I think most of the complexity comes from indexing (described below).

Here's an example of four shader binding tables stored consecutively: the table of ray generation shaders, the table of miss shaders, the table of hit groups (described below), and the table of callable shaders. These have one record, two records, one record, and three records, respectively.

![Four shader binding tables stored consecutively. From the [NVIDIA Vulkan Ray Tracing Callable Shaders - Tutorial](https://github.com/nvpro-samples/vk_raytracing_tutorial_KHR/tree/master/ray_tracing_callable).](images/e11-sbt.png)

The shader binding tables for each of the different types of groups don't need
to be stored consecutively, or even in the same `VkBuffer`.
Instead, `vkCmdTraceRaysKHR` takes `VkStridedDeviceAddressRegionKHR`s of memory
including a pointer to the start of each SBT in GPU memory.

There are four types of records. Each stores a shader group handle referencing up to three shaders and a shader record. Here are the shaders each record type points to:

* **Ray generation record**: Only a ray generation shader.
* **Miss record**: Only a miss shader.
* **Hit record**: A closest hit shader, an optional any hit shader, and an optional intersection shader (only for procedural hit groups).
* **Callable record**: Only a callable shader.

### Indexing into SBTs

When looking up a shader, different factors will tell the traversal engine the index of the shader to call. This is to support different shaders per instance, per geometry inside an instance, and per **ray type** (for instance, imagine if one wanted shadow rays to use less computationally expensive shaders than camera rays, e.g. without shading). These factors are:

1. `missIndex` from the GLSL `traceRayEXT` call;
2. The instance's `instanceShaderBindingTableRecordOffset` from TLAS creation;
3. `sbtRecordOffset` from the GLSL `traceRayEXT` call;
4. `sbtRecordStride` from the GLSL `traceRayEXT` call;
5. The [geometry index](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/html/chap39.html#acceleration-structure-geometry-index) `geometryIndex` of each geometry inside a BLAS.

Here's how SBT indexing works:

#### Ray Generation

This always uses the ray generation shader at index 0.
(See the "Exact Indexing Rules" section below for launching using a different
ray generation shader in the SBT.)

#### Miss

When a ray didn't intersect anything, traversal calls the index `missIndex` miss shader.

#### Intersection, Any Hit, and Closest Hit

In these sections of the ray tracing pipeline block diagram, traversal calls the corresponding shader from the hit record with index

<center>`instanceShaderBindingTableRecordOffset + sbtRecordOffset + sbtRecordStride * geometryIndex`.</center>

!!! Tip Exact Indexing Rules
    These rules are slightly simplified, since they assume that the
    `VkStridedDeviceAddressRegionKHR`s passed to the `vkCmdTraceRaysKHR`
    function all point to the start of their SBT region and that their `stride`
    members are indeed the strides between records.
    The Vulkan API specification defines SBT indexing in terms of device memory
    addresses, so if this isn't the case, then one must use the full rules.
    See the [Shader Binding Table](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/html/chap38.html#shader-binding-table)
    section in the Vulkan API specification.

Will Usher's [The RTX Shader Binding Table Three Ways](https://www.willusher.io/graphics/2019/11/20/the-sbt-three-ways) is another good reference for understanding SBTs across APIs.

You'll sometimes see a table of contiguous SBTs referred to as a single SBT.

### No Work Groups

In previous chapters, we arranged invocations in groups of 128 in work groups, and then arranged those work groups within a grid.

~~~ cpp
    // Run the compute shader with enough workgroups to cover the entire buffer:
    vkCmdDispatch(cmdBuffer, (render_width + WORKGROUP_WIDTH - 1) / WORKGROUP_WIDTH,
                  (render_height + WORKGROUP_HEIGHT - 1) / WORKGROUP_HEIGHT, 1);
~~~

Ray tracing pipelines remove work groups, giving the scheduler (that decides how to group threads into 32-thread warps/subgroups and when to run warps) much more flexibility, and allowing us to avoid some of the calculations we did in `vkCmdDispatch`. (Some readers may be familiar with subgroup operations in GLSL; these get a bit more complex. See the [GLSL_EXT_ray_tracing specification](https://github.com/KhronosGroup/GLSL/blob/master/extensions/ext/GLSL_EXT_ray_tracing.txt).)

~~~ cpp
    // Run the ray tracing pipeline and trace rays
    vkCmdTraceRaysKHR(cmdBuffer,           // Command buffer
                      ...
~~~ cpp highlight
                      render_width,        // Width of dispatch
                      render_height,       // Height of dispatch
                      1);                  // Depth of dispatch
~~~

This helps especially for complex shader networks, since the flexibility given
to the scheduling system can allow it to avoid some of the divergence from the
`switch` statement in the previous chapter!

![Thread scheduling for ray tracing pipelines. From the [NVIDIA Turing GPU Architecture whitepaper](https://www.nvidia.com/content/dam/en-zz/Solutions/design-visualization/technologies/turing-architecture/NVIDIA-Turing-Architecture-Whitepaper.pdf).](images/e11-rt-scheduling.png)

Now that we've covered the different aspects of ray tracing pipelines, let's see how to modify the code to use them!



## Ray Tracing Pipelines Without Ray Tracing

A ray generation shader is the starting point for work in a ray tracing
pipeline, and acts much like a compute shader. This means that we can create a
ray tracing pipeline without miss, hit, or callable groups, and use that to
mimic a compute pipeline. (In fact, an earlier version of this tutorial used ray
generation shaders instead of compute shaders entirely!) Later, we'll see how to
add more shaders to the ray tracing pipeline.

!!! Tip
    This chapter has a folder with reference code for each of three sections.
    You can find the reference code at the end of this section in the `checkpoints/e11_rt_pipeline_1` directory.
    If you run into any problems with the code in this section, you can diff your
    files against the reference files to see if there are any differences.

### This Section's Ray Generation Shader

First, delete `shaders/raytrace.comp.glsl` (or move it to a different folder). Replace it with
a new file in the `shaders` folder called `raytrace.rgen.glsl`, with the following code:

~~~ glsl
#version 460
#extension GL_EXT_ray_tracing : require
#extension GL_EXT_scalar_block_layout : require
#extension GL_GOOGLE_include_directive : require
#include "../common.h"

// Binding BINDING_IMAGEDATA in set 0 is a storage image with four 32-bit floating-point channels,
// defined using a uniform image2D variable.
layout(binding = BINDING_IMAGEDATA, set = 0, rgba32f) uniform image2D storageImage;

void main()
{
  // The resolution of the image, which is the same as the launch size:
  const ivec2 resolution = imageSize(storageImage);

  // Get the coordinates of the pixel for this invocation:
  //
  // .-------.-> x
  // |       |
  // |       |
  // '-------'
  // v
  // y
  const ivec2 pixel = ivec2(gl_LaunchIDEXT.xy);

  // If the pixel is outside of the image, don't do anything:
  if((pixel.x >= resolution.x) || (pixel.y >= resolution.y))
  {
    return;
  }
  
  // Set the color of the pixel `pixel` in the storage image as follows:
  vec4 color = vec4(pixel.x / float(resolution.x),  //
                    pixel.y / float(resolution.y),  //
                    0.0,                            //
                    0.0);
  imageStore(storageImage, pixel, color);
}
~~~

This is similar to the compute shader in [this chapter](index.html#descriptors/anewcomputeshader) of the main tutorial.
The main differences are that the `.rgen.glsl` extension tells `glslangValidator` that this is a ray generation shader,
and that we use `gl_LaunchIDEXT` instead of `gl_GlobalInvocationID` to get the invocation index.

### Switching Extensions

We'll need to use the `VK_KHR_ray_tracing_pipeline` extension and will remove the `VK_KHR_ray_query` extension.
To do this, replace

~~~ cpp
  VkPhysicalDeviceRayQueryFeaturesKHR rayQueryFeatures = nvvk::make<VkPhysicalDeviceRayQueryFeaturesKHR>();
  deviceInfo.addDeviceExtension(VK_KHR_RAY_QUERY_EXTENSION_NAME, false, &rayQueryFeatures);
~~~

in `main.cpp` with

~~~ cpp
  VkPhysicalDeviceRayTracingPipelineFeaturesKHR rtPipelineFeatures = nvvk::make<VkPhysicalDeviceRayTracingPipelineFeaturesKHR>();
  deviceInfo.addDeviceExtension(VK_KHR_RAY_TRACING_PIPELINE_EXTENSION_NAME, false, &rtPipelineFeatures);
~~~

We'll also need to check that the device supports ray tracing.

~~~ cpp
  // Device must support acceleration structures and ray tracing pipelines:
  assert(asFeatures.accelerationStructure == VK_TRUE && rtPipelineFeatures.rayTracingPipeline == VK_TRUE);
~~~

### Calculating SBT Stride

Applications can choose the stride between records in each shader binding table
they create. This allows applications to choose the size of each SBT's
shader records (the size of the data in each SBT record other than the shader group handle).

However, these strides must satisfy several conditions, which use values from
a

<center>[`VkPhysicalDeviceRayTracingPipelinePropertiesKHR`](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VkPhysicalDeviceRayTracingPipelinePropertiesKHR.html)</center>

object written by `vkGetPhysicalDeviceProperties2`:

- The stride must be greater than `shaderGroupHandleSize`. In other words, each SBT record must have enough space for its shader group handle.
- The stride must be a multiple of `shaderGroupHandleAlignment`.
- The stride must be less than `maxShaderGroupStride`.

In addition, each SBT must start at a multiple of `shaderGroupBaseAlignment` bytes.

Given these constraints, and that we want to store all SBTs contiguously in
a single buffer, we choose to

- Assert that `shaderGroupBaseAlignment` is a multiple of `shaderGroupHandleAlignment`;
- Round `shaderGroupHandleSize` up to a multiple of `shaderGroupBaseAlignment` to get the SBT stride;
- Assert that this result is less than `maxShaderGroupStride`.

To get the properties of ray tracing pipelines on this device, then to compute
the SBT stride in this way, add the following code:

~~~ cpp
  // Get the properties of ray tracing pipelines on this device. We do this by
  // using vkGetPhysicalDeviceProperties2, and extending this by chaining on a
  // VkPhysicalDeviceRayTracingPipelinePropertiesKHR object to get both
  // physical device properties and ray tracing pipeline properties.
  // This gives us information about shader binding tables.
  VkPhysicalDeviceRayTracingPipelinePropertiesKHR rtPipelineProperties =
      nvvk::make<VkPhysicalDeviceRayTracingPipelinePropertiesKHR>();
  VkPhysicalDeviceProperties2 physicalDeviceProperties = nvvk::make<VkPhysicalDeviceProperties2>();
  physicalDeviceProperties.pNext                       = &rtPipelineProperties;
  vkGetPhysicalDeviceProperties2(context.m_physicalDevice, &physicalDeviceProperties);
  const VkDeviceSize sbtHeaderSize      = rtPipelineProperties.shaderGroupHandleSize;
  const VkDeviceSize sbtBaseAlignment   = rtPipelineProperties.shaderGroupBaseAlignment;
  const VkDeviceSize sbtHandleAlignment = rtPipelineProperties.shaderGroupHandleAlignment;

  // Compute the stride between shader binding table (SBT) records.
  // This must be:
  // - Greater than rtPipelineProperties.shaderGroupHandleSize (since a record
  //     contains a shader group handle)
  // - A multiple of rtPipelineProperties.shaderGroupHandleAlignment
  // - Less than or equal to rtPipelineProperties.maxShaderGroupStride
  // In addition, each SBT must start at a multiple of
  // rtPipelineProperties.shaderGroupBaseAlignment.
  // Since we store all records contiguously in a single SBT, we assert that
  // sbtBaseAlignment is a multiple of sbtHandleAlignment, round sbtHeaderSize
  // up to a multiple of sbtBaseAlignment, and then assert that the result is
  // less than or equal to maxShaderGroupStride.
  assert(sbtBaseAlignment % sbtHandleAlignment == 0);
  const VkDeviceSize sbtStride = sbtBaseAlignment *  //
                                 ((sbtHeaderSize + sbtBaseAlignment - 1) / sbtBaseAlignment);
  assert(sbtStride <= rtPipelineProperties.maxShaderGroupStride);
~~~

Note that this can be a larger stride than necessary. On one system, for
instance, `sbtHeaderSize` was 32, `sbtBaseAlignment` was 64, and
`sbtHandleAlignment` was 32. In this case, it would have been possible to use a
stride of 32 bytes, so long as we made sure to align the start each SBT at a
multiple of 64 bytes. However, because it's slightly less code to use a common
alignment of 64 bytes, the code above chooses this latter value.

In an application where minimizing the size of the SBTs is important, we might
instead compute the necessary stride for each SBT by rounding up the needed
record size (shader group handle size plus shader record size) to a multiple of
`shaderGroupHandleSize`, then compute the size of each SBT, then compute the
size of the buffer that would be needed if each SBT started at a multiple of
`shaderGroupBaseAlignment` bytes.

### Setting All Instances' Shader Binding Table Offsets to 0

In this subsection, we want all instances to use the closest-hit shader with
index 0 instead of trying to randomly access indices 1 through 8, which don't
exist yet.

Replace the line that sets `instance.instanceShaderBindingTableRecordOffset` with

~~~ cpp
      // An offset that will be added when looking up the instance's shader in the SBT.
      instance.instanceShaderBindingTableRecordOffset = 0;
~~~

This is important; without it, one might get an image like below, or one might
get a device lost error.

![An example of a glitchy image where 8/9 instances had SBT offsets that resulted in indexing outside the SBT.](images/e11-sbtGlitch.png)

### Modifying Usage

Previously, we told `nvvk::DescriptorSetContainer` that each of the descriptors would be used by
the compute shader. Change this to the ray generation shader:

~~~ cpp
  descriptorSetContainer.addBinding(BINDING_IMAGEDATA, VK_DESCRIPTOR_TYPE_STORAGE_IMAGE, 1, VK_SHADER_STAGE_RAYGEN_BIT_KHR);
  descriptorSetContainer.addBinding(BINDING_TLAS, VK_DESCRIPTOR_TYPE_ACCELERATION_STRUCTURE_KHR, 1, VK_SHADER_STAGE_RAYGEN_BIT_KHR);
  descriptorSetContainer.addBinding(BINDING_VERTICES, VK_DESCRIPTOR_TYPE_STORAGE_BUFFER, 1, VK_SHADER_STAGE_CLOSEST_HIT_BIT_KHR);
  descriptorSetContainer.addBinding(BINDING_INDICES, VK_DESCRIPTOR_TYPE_STORAGE_BUFFER, 1, VK_SHADER_STAGE_CLOSEST_HIT_BIT_KHR);
~~~

Also change `pushConstantRange.stageFlags` to list the flag for the ray generation shader stage:

~~~ cpp
  pushConstantRange.stageFlags = VK_SHADER_STAGE_RAYGEN_BIT_KHR;
~~~

### Creating the Ray Tracing Pipeline and SBTs

We'll use six steps to create the ray tracing pipeline and SBTs (here stored contiguously) together:

1. Load SPIR-V code for each of the shader modules.
2. Create an array of "shader stages", `VkPipelineShaderStageCreateInfo` structures, pointing to a shader module, a shader entry point (like `main()`), and listing a shader stage using `VkShaderStageFlags`. This is because SPIR-V shader modules can contain multiple entry points; although GLSL doesn't support this when translating to SPIR-V, HLSL, slang, Circle-C++, and rust-gpu all do.
3. Create an array of shader groups, which correspond to the entries in the SBT. Each `VkRayTracingShaderGroupCreateInfoKHR` includes the indexes of one or more elements in the `VkPipelineShaderStageCreateInfo` array.
4. Create the ray tracing pipeline from these two arrays.
5. Get the shader group handles from the ray tracing pipeline, and copy them into an array with a stride of `sbtStride`,
6. Create a `VkBuffer` with the contents of that array. This contains the SBTs.

To do this, replace the previous code for creating the compute pipeline:

~~~ cpp
  VkShaderModule rayTraceModule =
      nvvk::createShaderModule(context, nvh::loadFile("shaders/raytrace.comp.glsl.spv", true, searchPaths));
  debugUtil.setObjectName(rayTraceModule, "rayTraceModule");

  // Describes the entrypoint and the stage to use for this shader module in the pipeline
  VkPipelineShaderStageCreateInfo shaderStageCreateInfo = nvvk::make<VkPipelineShaderStageCreateInfo>();
  shaderStageCreateInfo.stage                           = VK_SHADER_STAGE_COMPUTE_BIT;
  shaderStageCreateInfo.module                          = rayTraceModule;
  shaderStageCreateInfo.pName                           = "main";

  // Create the compute pipeline
  VkComputePipelineCreateInfo pipelineCreateInfo = nvvk::make<VkComputePipelineCreateInfo>();
  pipelineCreateInfo.stage                       = shaderStageCreateInfo;
  pipelineCreateInfo.layout                      = descriptorSetContainer.getPipeLayout();
  // Don't modify flags, basePipelineHandle, or basePipelineIndex
  VkPipeline computePipeline;
  NVVK_CHECK(vkCreateComputePipelines(context,                 // Device
                                      VK_NULL_HANDLE,          // Pipeline cache (uses default)
                                      1, &pipelineCreateInfo,  // Compute pipeline create info
                                      nullptr,                 // Allocator (uses default)
                                      &computePipeline));      // Output
  debugUtil.setObjectName(computePipeline, "computePipeline");
~~~

with the following:

~~~ cpp
  VkShaderModule rayGenModule =
      nvvk::createShaderModule(context, nvh::loadFile("shaders/raytrace.rgen.glsl.spv", true, searchPaths));
  debugUtil.setObjectName(rayGenModule, "rayGenModule");

  // Create the shader binding table and ray tracing pipeline.
  // We'll create the ray tracing pipeline by specifying the shaders + layout,
  // and then get the handles of the shaders for the shader binding table from
  // the pipeline.
  VkPipeline   rtPipeline;
  nvvk::Buffer rtSBTBuffer;  // The buffer for the Shader Binding Table
  {
    // First, we create objects that point to each of our shaders.
    // These are called "shader stages" in this context.
    // These are shader module + entry point + stage combinations, because each
    // shader module can contain multiple entry points (e.g. main1, main2...)
    std::array<VkPipelineShaderStageCreateInfo, 1> stages;  // Pointers to shaders

    // Stage 0 will be the raygen shader.
    stages[0]        = nvvk::make<VkPipelineShaderStageCreateInfo>();
    stages[0].stage  = VK_SHADER_STAGE_RAYGEN_BIT_KHR;  // Kind of shader
    stages[0].module = rayGenModule;                    // Contains the shader
    stages[0].pName  = "main";                          // Name of the entry point

    // Then we make groups point to the shader stages. Each group can point to
    // one or two shader stages depending on the type, by specifying the index
    // in the stages array. These groups of handles then become the most
    // important part of the entries in the shader binding table.
    // Stores the indices of stages in each group:
    std::array<VkRayTracingShaderGroupCreateInfoKHR, 1> groups;

    // The vkCmdTraceRays call will eventually refer to ray gen, miss, hit, and
    // callable shader binding tables and ranges.
    // A VK_RAY_TRACING_SHADER_GROUP_TYPE_GENERAL_KHR group type is for a group
    // of one shader (a ray gen shader in a ray gen SBT region, a miss shader in
    // a miss SBT region, and so on.)
    // A VK_RAY_TRACING_SHADER_GROUP_TYPE_TRIANGLES_HIT_GROUP_KHR group type
    // is for an instance containing triangles. It can point to closest hit and
    // any hit shaders.
    // A VK_RAY_TRACING_SHADER_GROUP_TYPE_PROCEDURAL_HIT_GROUP_KHR group type
    // is for a procedural instance, and can point to an intersection, any hit,
    // and closest hit shader.

    // We lay out our shader binding table like this:
    // RAY GEN REGION
    // Group 0 - points to Stage 0
    groups[0]               = nvvk::make<VkRayTracingShaderGroupCreateInfoKHR>();
    groups[0].type          = VK_RAY_TRACING_SHADER_GROUP_TYPE_GENERAL_KHR;
    groups[0].generalShader = 0;  // Index of ray gen, miss, or callable in `stages`

    // Now, describe the ray tracing pipeline, ike creating a compute pipeline:
    VkRayTracingPipelineCreateInfoKHR pipelineCreateInfo = nvvk::make<VkRayTracingPipelineCreateInfoKHR>();
    pipelineCreateInfo.flags                             = 0;  // No flags to set
    pipelineCreateInfo.stageCount                        = static_cast<uint32_t>(stages.size());
    pipelineCreateInfo.pStages                           = stages.data();
    pipelineCreateInfo.groupCount                        = static_cast<uint32_t>(groups.size());
    pipelineCreateInfo.pGroups                           = groups.data();
    pipelineCreateInfo.maxPipelineRayRecursionDepth      = 1;  // Depth of call tree
    pipelineCreateInfo.layout                            = descriptorSetContainer.getPipeLayout();
    NVVK_CHECK(vkCreateRayTracingPipelinesKHR(context,                 // Device
                                              VK_NULL_HANDLE,          // Deferred operation or VK_NULL_HANDLE
                                              VK_NULL_HANDLE,          // Pipeline cache or VK_NULL_HANDLE
                                              1, &pipelineCreateInfo,  // Array of create infos
                                              nullptr,                 // Allocator
                                              &rtPipeline));
    debugUtil.setObjectName(rtPipeline, "rtPipeline");

    // Now create and write the shader binding table, by getting the shader
    // group handles from the ray tracing pipeline and writing them into a
    // Vulkan buffer object.

    // Get the shader group handles:
    std::vector<uint8_t> cpuShaderHandleStorage(sbtHeaderSize * groups.size());
    NVVK_CHECK(vkGetRayTracingShaderGroupHandlesKHR(context,                               // Device
                                                    rtPipeline,                            // Pipeline
                                                    0,                                     // First group
                                                    static_cast<uint32_t>(groups.size()),  // Number of groups
                                                    cpuShaderHandleStorage.size(),         // Size of buffer
                                                    cpuShaderHandleStorage.data()));       // Data buffer
    // Allocate the shader binding table. We get its device address, and
    // use it as a shader binding table. As before, we set its memory property
    // flags so that it can be read and written from the CPU.
    const uint32_t sbtSize = static_cast<uint32_t>(sbtStride * groups.size());
    rtSBTBuffer            = allocator.createBuffer(
        sbtSize, VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT | VK_BUFFER_USAGE_SHADER_BINDING_TABLE_BIT_KHR,
        VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT | VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | VK_MEMORY_PROPERTY_HOST_COHERENT_BIT);
    debugUtil.setObjectName(rtSBTBuffer.buffer, "rtSBTBuffer");
    // Copy the shader group handles to the SBT:
    uint8_t* mappedSBT = reinterpret_cast<uint8_t*>(allocator.map(rtSBTBuffer));
    for(size_t groupIndex = 0; groupIndex < groups.size(); groupIndex++)
    {
      memcpy(&mappedSBT[groupIndex * sbtStride], &cpuShaderHandleStorage[groupIndex * sbtHeaderSize], sbtHeaderSize);
    }
    allocator.unmap(rtSBTBuffer);
    // Clean up:
    allocator.finalizeAndReleaseStaging();
  }
~~~

### Pointing to SBT Regions

`VkCmdTraceRaysKHR` uses `VkStridedDeviceAddressregionKHR` objects to say where each block of shaders is held in memory. These could change per draw call, but let's create them up front since they're the same every time here. Add the following code:

~~~ cpp
  VkStridedDeviceAddressRegionKHR sbtRayGenRegion, sbtMissRegion, sbtHitRegion, sbtCallableRegion;
  const VkDeviceAddress           sbtStartAddress = GetBufferDeviceAddress(context, rtSBTBuffer.buffer);
  {
    // The ray generation shader region:
    sbtRayGenRegion.deviceAddress = sbtStartAddress;  // Starts here
    sbtRayGenRegion.stride        = sbtStride;        // Uses this stride
    sbtRayGenRegion.size          = sbtStride;        // Is this number of bytes long (1 group)

    sbtMissRegion      = sbtRayGenRegion;  // The miss shader region:
    sbtMissRegion.size = 0;                // Is empty

    sbtHitRegion      = sbtRayGenRegion;  // The hit group region:
    sbtHitRegion.size = 0;                // Is empty

    sbtCallableRegion      = sbtRayGenRegion;  // The callable shader region:
    sbtCallableRegion.size = 0;                // Is empty
  }
~~~

### Launching a Ray Tracing Pipeline

Next, bind the command buffer, update the `vkCmdBindDescriptorSets` and `vkCmdPushConstants` calls to use the ray generation shader stage, and call `vkCmdTraceRaysKHR` instead of `vkCmdDispatch` to launch the ray tracing pipeline.

To do this, replace the lines from `vkCmdBindPipeline` to `vkCmdDispatch` with the following:

~~~ cpp
    // Bind the ray tracing pipeline:
    vkCmdBindPipeline(cmdBuffer, VK_PIPELINE_BIND_POINT_RAY_TRACING_KHR, rtPipeline);
    // Bind the descriptor set
    VkDescriptorSet descriptorSet = descriptorSetContainer.getSet(0);
    vkCmdBindDescriptorSets(cmdBuffer, VK_PIPELINE_BIND_POINT_RAY_TRACING_KHR, descriptorSetContainer.getPipeLayout(),
                            0, 1, &descriptorSet, 0, nullptr);

    // Push push constants:
    pushConstants.sample_batch = sampleBatch;
    vkCmdPushConstants(cmdBuffer,                               // Command buffer
                       descriptorSetContainer.getPipeLayout(),  // Pipeline layout
                       VK_SHADER_STAGE_RAYGEN_BIT_KHR,          // Stage flags
                       0,                                       // Offset
                       sizeof(PushConstants),                   // Size in bytes
                       &pushConstants);                         // Data

    // Run the ray tracing pipeline and trace rays
    vkCmdTraceRaysKHR(cmdBuffer,           // Command buffer
                      &sbtRayGenRegion,    // Region of memory with ray generation groups
                      &sbtMissRegion,      // Region of memory with miss groups
                      &sbtHitRegion,       // Region of memory with hit groups
                      &sbtCallableRegion,  // Region of memory with callable groups
                      render_width,        // Width of dispatch
                      render_height,       // Height of dispatch
                      1);                  // Depth of dispatch
~~~

### Cleaning Up

Finally, replace the `vkDestroyPipeline` and `vkDestroyShaderModule` calls with

~~~ cpp
  allocator.destroy(rtSBTBuffer);
  vkDestroyPipeline(context, rtPipeline, nullptr);
  vkDestroyShaderModule(context, rayGenModule, nullptr);
~~~

If all went well, you should now be able to run the program and get the following image, which it renders 32 times:

![The output of this section's ray generation shader, which acts like a compute shader.](images/e11-output-1.png)



## One Material

In this section, we'll add a closest hit shader, which will act like `material0` in the previous chapter, and a miss shader, which will return the color of the sky. While the previous section included more C++ code modifications, this section will largely involve refactoring the GLSL code.

!!! Tip
    This chapter has a folder with reference code for each of three sections.
    You can find the reference code at the end of this section in the `checkpoints/e11_rt_pipeline_2` directory.
    If you run into any problems with the code in this section, you can diff your
    files against the reference files to see if there are any differences.

### A Common File for Closest-Hit Shaders

Let's move some of the code that will only be used by closest-hit shaders out from `shaderCommon.h` to its own file, `shaders/closestHitCommon.h`.

Create a new file called `closestHitCommon.h` in the `shaders` subdirectory, and fill it with the following:

~~~ glsl
// Common file to make closest-hit GLSL shaders shorter to write.
// At the moment, each .glsl file can only have a single entry point, even
// though SPIR-V supports multiple entry points per module - this is why
// we have many small .rchit.glsl files.
#ifndef VK_MINI_PATH_TRACER_CLOSEST_HIT_COMMON_H
#define VK_MINI_PATH_TRACER_CLOSEST_HIT_COMMON_H

#extension GL_EXT_ray_tracing : require
#extension GL_EXT_scalar_block_layout : require
#include "../common.h"
#include "shaderCommon.h"

// This will store two of the barycentric coordinates of the intersection when
// closest-hit shaders are called:
hitAttributeEXT vec2 attributes;

// These shaders can access the vertex and index buffers:
// The scalar layout qualifier here means to align types according to the alignment
// of their scalar components, instead of e.g. padding them to std140 rules.
layout(binding = BINDING_VERTICES, set = 0, scalar) buffer Vertices
{
  vec3 vertices[];
};
layout(binding = BINDING_INDICES, set = 0, scalar) buffer Indices
{
  uint indices[];
};

// The payload:
layout(location = 0) rayPayloadInEXT PassableInfo pld;

struct HitInfo
{
  vec3 objectPosition;
  vec3 worldPosition;
  vec3 worldNormal;
  vec3 color;
};

// Gets hit info about the object at the intersection. This uses GLSL variables
// defined in closest hit stages instead of ray queries.
HitInfo getObjectHitInfo()
{
  HitInfo result;
  // Get the ID of the triangle
  const int primitiveID = gl_PrimitiveID;

  // Get the indices of the vertices of the triangle
  const uint i0 = indices[3 * primitiveID + 0];
  const uint i1 = indices[3 * primitiveID + 1];
  const uint i2 = indices[3 * primitiveID + 2];

  // Get the vertices of the triangle
  const vec3 v0 = vertices[i0];
  const vec3 v1 = vertices[i1];
  const vec3 v2 = vertices[i2];


  // Get the barycentric coordinates of the intersection
  vec3 barycentrics = vec3(0.0, attributes.x, attributes.y);
  barycentrics.x    = 1.0 - barycentrics.y - barycentrics.z;

  // Compute the coordinates of the intersection
  result.objectPosition = v0 * barycentrics.x + v1 * barycentrics.y + v2 * barycentrics.z;
  // Transform from object space to world space:
  result.worldPosition = gl_ObjectToWorldEXT * vec4(result.objectPosition, 1.0f);


  // Compute the normal of the triangle in object space, using the right-hand rule:
  //    v2      .
  //    |\      .
  //    | \     .
  //    |/ \    .
  //    /   \   .
  //   /|    \  .
  //  L v0---v1 .
  // n
  const vec3 objectNormal = cross(v1 - v0, v2 - v0);
  // Transform normals from object space to world space. These use the transpose of the inverse matrix,
  // because they're directions of normals, not positions:
  result.worldNormal = normalize((objectNormal * gl_WorldToObjectEXT).xyz);

  // Flip the normal so it points against the ray direction:
  const vec3 rayDirection = gl_WorldRayDirectionEXT;
  result.worldNormal      = faceforward(result.worldNormal, rayDirection, result.worldNormal);

  return result;
}

// offsetPositionAlongNormal shifts a point on a triangle surface so that a
// ray bouncing off the surface with tMin = 0.0 is no longer treated as
// intersecting the surface it originated from.
//
// Here's the old implementation of it we used in earlier chapters:
// vec3 offsetPositionAlongNormal(vec3 worldPosition, vec3 normal)
// {
//   return worldPosition + 0.0001 * normal;
// }
//
// However, this code uses an improved technique by Carsten Wächter and
// Nikolaus Binder from "A Fast and Robust Method for Avoiding
// Self-Intersection" from Ray Tracing Gems (version 1.7, 2020).
// The normal can be negated if one wants the ray to pass through
// the surface instead.
vec3 offsetPositionAlongNormal(vec3 worldPosition, vec3 normal)
{
  // Convert the normal to an integer offset.
  const float int_scale = 256.0f;
  const ivec3 of_i      = ivec3(int_scale * normal);

  // Offset each component of worldPosition using its binary representation.
  // Handle the sign bits correctly.
  const vec3 p_i = vec3(  //
      intBitsToFloat(floatBitsToInt(worldPosition.x) + ((worldPosition.x < 0) ? -of_i.x : of_i.x)),
      intBitsToFloat(floatBitsToInt(worldPosition.y) + ((worldPosition.y < 0) ? -of_i.y : of_i.y)),
      intBitsToFloat(floatBitsToInt(worldPosition.z) + ((worldPosition.z < 0) ? -of_i.z : of_i.z)));

  // Use a floating-point offset instead for points near (0,0,0), the origin.
  const float origin     = 1.0f / 32.0f;
  const float floatScale = 1.0f / 65536.0f;
  return vec3(  //
      abs(worldPosition.x) < origin ? worldPosition.x + floatScale * normal.x : p_i.x,
      abs(worldPosition.y) < origin ? worldPosition.y + floatScale * normal.y : p_i.y,
      abs(worldPosition.z) < origin ? worldPosition.z + floatScale * normal.z : p_i.z);
}

vec3 diffuseReflection(vec3 normal, inout uint rngState)
{
  // For a random diffuse bounce direction, we follow the approach of
  // Ray Tracing in One Weekend, and generate a random point on a sphere
  // of radius 1 centered at the normal. This uses the random_unit_vector
  // function from chapter 8.5:
  const float theta     = 2.0 * k_pi * stepAndOutputRNGFloat(rngState);  // Random in [0, 2pi]
  const float u         = 2.0 * stepAndOutputRNGFloat(rngState) - 1.0;   // Random in [-1, 1]
  const float r         = sqrt(1.0 - u * u);
  const vec3  direction = normal + vec3(r * cos(theta), r * sin(theta), u);

  // Then normalize the ray direction:
  return normalize(direction);
}

#endif  // #ifndef VK_MINI_PATH_TRACER_CLOSEST_HIT_COMMON_H
~~~


### A Shorter `shaderCommon.h`

Now replace `shaderCommon.h` with the following, which will be used by all shaders:

~~~ glsl
// Common GLSL file shared across ray tracing shaders.
#ifndef VK_MINI_PATH_TRACER_SHADER_COMMON_H
#define VK_MINI_PATH_TRACER_SHADER_COMMON_H

struct PassableInfo
{
  vec3 color;         // The reflectivity of the surface.
  vec3 rayOrigin;     // The new ray origin in world-space.
  vec3 rayDirection;  // The new ray direction in world-space.
  uint rngState;      // State of the random number generator.
  bool rayHitSky;     // True if the ray hit the sky.
};

// Steps the RNG and returns a floating-point value between 0 and 1 inclusive.
float stepAndOutputRNGFloat(inout uint rngState)
{
  // Condensed version of pcg_output_rxs_m_xs_32_32, with simple conversion to floating-point [0,1].
  rngState  = rngState * 747796405 + 1;
  uint word = ((rngState >> ((rngState >> 28) + 4)) ^ rngState) * 277803737;
  word      = (word >> 22) ^ word;
  return float(word) / 4294967295.0f;
}

const float k_pi = 3.14159265;

#endif  // #ifndef VK_MINI_PATH_TRACER_SHADER_COMMON_H
~~~


### The Sky Miss Shader

When a ray misses all the geometry in the scene, traversal will run the miss shader (in this case, there's only one). Add a new file to the `shaders` subdirectory named `raytrace.rmiss.glsl` with the following code:

~~~ glsl
#version 460
#extension GL_EXT_ray_tracing : require
#extension GL_GOOGLE_include_directive : require

#include "shaderCommon.h"

// The payload:
layout(location = 0) rayPayloadInEXT PassableInfo pld;

void main() {
  // Returns the color of the sky in a given direction (in linear color space)
  // +y in world space is up, so:
  const float rayDirY = gl_WorldRayDirectionEXT.y;
  if(rayDirY > 0.0f)
  {
    pld.color = mix(vec3(1.0f), vec3(0.25f, 0.5f, 1.0f), rayDirY);
  }
  else
  {
    pld.color = vec3(0.03f);
  }

  pld.rayHitSky = true;
}
~~~


### The Material 0 Closest Hit Shader

For the material with index 0, add a new file to the `shaders` subdirectory named `material0.rchit.glsl` with the following code:

~~~ glsl
#version 460
#extension GL_GOOGLE_include_directive : require
#include "closestHitCommon.h"

void main()
{
  HitInfo hitInfo = getObjectHitInfo();

  pld.color        = vec3(0.7);
  pld.rayOrigin    = offsetPositionAlongNormal(hitInfo.worldPosition, hitInfo.worldNormal);
  pld.rayDirection = diffuseReflection(hitInfo.worldNormal, pld.rngState);
  pld.rayHitSky    = false;
}
~~~


### Path Tracing with Ray Tracing Pipelines

Replace `raytrace.rgen.glsl` with the following code. This acts like previous
chapters' path tracer, but the code to handle ray queries has been replaced by a
`traceRayEXT` call and reading the payload `pld`. We set the SBT record offset
and stride to 0, since we aren't using the relevant features.

~~~ glsl
#version 460
#extension GL_EXT_ray_tracing : require
#extension GL_EXT_scalar_block_layout : require
#extension GL_GOOGLE_include_directive : require
#include "../common.h"
#include "shaderCommon.h"

// Binding BINDING_IMAGEDATA in set 0 is a storage image with four 32-bit floating-point channels,
// defined using a uniform image2D variable.
layout(binding = BINDING_IMAGEDATA, set = 0, rgba32f) uniform image2D storageImage;
layout(binding = BINDING_TLAS, set = 0) uniform accelerationStructureEXT tlas;

layout(push_constant) uniform PushConsts
{
  PushConstants pushConstants;
};

// Ray payloads are used to send information between shaders.
layout(location = 0) rayPayloadEXT PassableInfo pld;

// Uses the Box-Muller transform to return a normally distributed (centered
// at 0, standard deviation 1) 2D point.
vec2 randomGaussian(inout uint rngState)
{
  // Almost uniform in (0,1] - make sure the value is never 0:
  const float u1    = max(1e-38, stepAndOutputRNGFloat(rngState));
  const float u2    = stepAndOutputRNGFloat(rngState);  // In [0, 2pi]
  const float r     = sqrt(-2.0 * log(u1));
  const float theta = 2 * k_pi * u2;  // Random in [0, 2pi]
  return r * vec2(cos(theta), sin(theta));
}

void main()
{
  // The resolution of the image, which is the same as the launch size:
  const ivec2 resolution = imageSize(storageImage);

  // Get the coordinates of the pixel for this invocation:
  //
  // .-------.-> x
  // |       |
  // |       |
  // '-------'
  // v
  // y
  const ivec2 pixel = ivec2(gl_LaunchIDEXT.xy);

  // If the pixel is outside of the image, don't do anything:
  if((pixel.x >= resolution.x) || (pixel.y >= resolution.y))
  {
    return;
  }

  // State of the random number generator with an initial seed.
  pld.rngState = uint((pushConstants.sample_batch * resolution.y + pixel.y) * resolution.x + pixel.x);

  // This scene uses a right-handed coordinate system like the OBJ file format, where the
  // +x axis points right, the +y axis points up, and the -z axis points into the screen.
  // The camera is located at (-0.001, 0, 53).
  const vec3 cameraOrigin = vec3(-0.001, 0.0, 53.0);
  // Define the field of view by the vertical slope of the topmost rays:
  const float fovVerticalSlope = 1.0 / 5.0;

  // The sum of the colors of all of the samples.
  vec3 summedPixelColor = vec3(0.0);

  // Limit the kernel to trace at most 64 samples.
  const int NUM_SAMPLES = 64;
  for(int sampleIdx = 0; sampleIdx < NUM_SAMPLES; sampleIdx++)
  {
    // Rays always originate at the camera for now. In the future, they'll
    // bounce around the scene.
    vec3 rayOrigin = cameraOrigin;
    // Compute the direction of the ray for this pixel. To do this, we first
    // transform the screen coordinates to look like this, where a is the
    // aspect ratio (width/height) of the screen:
    //           1
    //    .------+------.
    //    |      |      |
    // -a + ---- 0 ---- + a
    //    |      |      |
    //    '------+------'
    //          -1
    // Use a Gaussian with standard deviation 0.375 centered at the center of
    // the pixel:
    const vec2 randomPixelCenter = vec2(pixel) + vec2(0.5) + 0.375 * randomGaussian(pld.rngState);
    const vec2 screenUV          = vec2((2.0 * randomPixelCenter.x - resolution.x) / resolution.y,    //
                               -(2.0 * randomPixelCenter.y - resolution.y) / resolution.y);  // Flip the y axis
    // Create a ray direction:
    vec3 rayDirection = vec3(fovVerticalSlope * screenUV.x, fovVerticalSlope * screenUV.y, -1.0);
    rayDirection      = normalize(rayDirection);

    vec3 accumulatedRayColor = vec3(1.0);  // The amount of light that made it to the end of the current ray.

    // Limit the kernel to trace at most 32 segments.
    for(int tracedSegments = 0; tracedSegments < 32; tracedSegments++)
    {
      // Trace the ray into the scene and get data back!
      traceRayEXT(tlas,                  // Top-level acceleration structure
                  gl_RayFlagsOpaqueEXT,  // Ray flags, here saying "treat all geometry as opaque"
                  0xFF,                  // 8-bit instance mask, here saying "trace against all instances"
                  0,                     // SBT record offset
                  0,                     // SBT record stride for offset
                  0,                     // Miss index
                  rayOrigin,             // Ray origin
                  0.0,                   // Minimum t-value
                  rayDirection,          // Ray direction
                  10000.0,               // Maximum t-value
                  0);                    // Location of payload

      // Compute the amount of light that returns to this sample from the ray
      accumulatedRayColor *= pld.color;

      if(pld.rayHitSky)
      {
        // Done tracing this ray.
        // Sum this with the pixel's other samples.
        // (Note that we treat a ray that didn't find a light source as if it had
        // an accumulated color of (0, 0, 0)).
        summedPixelColor += accumulatedRayColor;

        break;
      }
      else
      {
        // Start a new segment
        rayOrigin    = pld.rayOrigin;
        rayDirection = pld.rayDirection;
      }
    }
  }

  // Blend with the averaged image in the buffer:
  vec3 averagePixelColor = summedPixelColor / float(NUM_SAMPLES);
  if(pushConstants.sample_batch != 0)
  {
    // Read the storage image:
    const vec3 previousAverageColor = imageLoad(storageImage, pixel).rgb;
    // Compute the new average:
    averagePixelColor =
        (pushConstants.sample_batch * previousAverageColor + averagePixelColor) / (pushConstants.sample_batch + 1);
  }
  // Set the color of the pixel `pixel` in the storage image to `averagePixelColor`:
  imageStore(storageImage, pixel, vec4(averagePixelColor, 0.0));
}
~~~


### Modifying the C++ Code

In `main.cpp`, we'll need to load three SPIR-V shader modules instead of one. I'm using an `std::array` for this, and defined the `NUM_C_HIT_SHADERS` constant to keep track of the number of closest hit shaders in one place (since this will change in the next chapter):

~~~ cpp
  // Shader loading and pipeline creation
~~~ cpp highlight
  const size_t                                      NUM_C_HIT_SHADERS = 1;
  std::array<VkShaderModule, 2 + NUM_C_HIT_SHADERS> modules;
  modules[0] = nvvk::createShaderModule(context, nvh::loadFile("shaders/raytrace.rgen.glsl.spv", true, searchPaths));
  debugUtil.setObjectName(modules[0], "Ray generation module (raytrace.rgen.glsl.spv)");
  modules[1] = nvvk::createShaderModule(context, nvh::loadFile("shaders/raytrace.rmiss.glsl.spv", true, searchPaths));
  debugUtil.setObjectName(modules[1], "Miss module (raytrace.rmiss.glsl.spv)");
  modules[2] = nvvk::createShaderModule(context, nvh::loadFile("shaders/material0.rchit.glsl.spv", true, searchPaths));
  debugUtil.setObjectName(modules[2], "Material 0 shader module");
~~~

Then modify the code for the `VkPipelineShaderStageCreateInfo` structs to list all three shader stages as follows:

~~~ cpp
    std::array<VkPipelineShaderStageCreateInfo, 2 + NUM_C_HIT_SHADERS> stages;  // Pointers to shaders

    // Stage 0 will be the raygen shader.
    stages[0]        = nvvk::make<VkPipelineShaderStageCreateInfo>();
    stages[0].stage  = VK_SHADER_STAGE_RAYGEN_BIT_KHR;  // Kind of shader
    stages[0].module = modules[0];                      // Contains the shader
    stages[0].pName  = "main";                          // Name of the entry point
    // Stage 1 will be the miss shader.
    stages[1]        = stages[0];
    stages[1].stage  = VK_SHADER_STAGE_MISS_BIT_KHR;  // Kind of shader
    stages[1].module = modules[1];                    // Contains the shader
    // Stage 2 will be the closest-hit shader.
    stages[2]        = stages[0];
    stages[2].stage  = VK_SHADER_STAGE_CLOSEST_HIT_BIT_KHR;  // Kind of shader
    stages[2].module = modules[2];                           // Contains the shader
~~~

Change the `groups` array to have a different length:

~~~ cpp
    std::array<VkRayTracingShaderGroupCreateInfoKHR, 2 + NUM_C_HIT_SHADERS> groups;
~~~

Then add two groups for the miss and closest hit shader to the SBT:

~~~ cpp
    // We lay out our shader binding table like this:
    // RAY GEN REGION
    // Group 0 - points to Stage 0
    groups[0]               = nvvk::make<VkRayTracingShaderGroupCreateInfoKHR>();
    groups[0].type          = VK_RAY_TRACING_SHADER_GROUP_TYPE_GENERAL_KHR;
    groups[0].generalShader = 0;  // Index of ray gen, miss, or callable in `stages`
~~~ cpp highlight
    // MISS SHADER REGION
    // Group 1 - points to Stage 1
    groups[1]               = nvvk::make<VkRayTracingShaderGroupCreateInfoKHR>();
    groups[1].type          = VK_RAY_TRACING_SHADER_GROUP_TYPE_GENERAL_KHR;
    groups[1].generalShader = 1;  // Index of ray gen, miss, or callable in `stages`
    // CLOSEST-HIT REGION
    // Group 2 - uses Stage 2 as its closest-hit shader
    groups[2]                  = nvvk::make<VkRayTracingShaderGroupCreateInfoKHR>();
    groups[2].type             = VK_RAY_TRACING_SHADER_GROUP_TYPE_TRIANGLES_HIT_GROUP_KHR;
    groups[2].closestHitShader = 2;  // Index of closest hit in `stages`
~~~

Then modify the `VkStridedDeviceAddressRegionKHR` code to point to the new regions
in the SBT:

~~~ cpp
    // The ray generation shader region:
    sbtRayGenRegion.deviceAddress = sbtStartAddress;  // Starts here
    sbtRayGenRegion.stride        = sbtStride;        // Uses this stride
    sbtRayGenRegion.size          = sbtStride;        // Is this number of bytes long (1 group)
~~~ cpp highlight

    sbtMissRegion               = sbtRayGenRegion;              // The miss shader region:
    sbtMissRegion.deviceAddress = sbtStartAddress + sbtStride;  // Starts sbtStride bytes (1 group) in
    sbtMissRegion.size          = sbtStride;                    // Is this number of bytes long (1 group)

    sbtHitRegion               = sbtRayGenRegion;                  // The hit group region:
    sbtHitRegion.deviceAddress = sbtStartAddress + 2 * sbtStride;  // Starts 2 * sbtStride bytes (2 groups) in
    sbtHitRegion.size          = sbtStride * NUM_C_HIT_SHADERS;    // Is this number of bytes long
~~~ cpp

    sbtCallableRegion      = sbtRayGenRegion;  // The callable shader region:
    sbtCallableRegion.size = 0;                // Is empty
~~~

Finally, clean up the array of shader modules by replacing the `vkDestroyShaderModule` call with

~~~ cpp
  for(VkShaderModule& shaderModule : modules)
  {
    vkDestroyShaderModule(context, shaderModule, nullptr);
  }
~~~

You should now be able to build and run the program and see all 441 instances with the same diffuse gray shader!

![441 instances with their parameters all pointing to the `material0` shader](images/e11-output-2.png)


## Multiple Materials

In this chapter, we'll add the other eight closest hit shaders, and make each instance point to a random closest hit shader!

!!! Tip
    This chapter has a folder with reference code for each of three sections.
    You can find the reference code at the end of this section in the `checkpoints/e11_rt_pipeline_3` directory.
    If you run into any problems with the code in this section, you can diff your
    files against the reference files to see if there are any differences.

### Get Materials 1 Through 8

As before, I've written nine total closest hit shaders, each representing a different material. Since these are similar to the ones in previous chapters, download the files `material1.rchit.glsl` through `material8.rchit.glsl` from [the section's Checkpoints folder](https://github.com/nvpro-samples/vk_mini_path_tracer/tree/main/checkpoints/e11_rt_pipeline_3/shaders) and place them in the project's `shaders` subdirectory next to `material0.rchit.glsl`.

!!! Tip Why Does This Use a Different GLSL and SPIR-V File per Material?
    GLSL requires that each file have only one entry point, named `main`. There are a few other things this chapter could do, but I held off on them because I didn't want to introduce more new things than necessary in this chapter:
    
    - We could use a library such as `shaderc` to programmatically generate SPIR-V code from GLSL code as needed (`nvvk::ShaderModuleManager` in the nvpro-samples is a helper function for using `shaderc`).
    - We could have the build system compile each GLSL file into a SPIR-V file, then rename each of the entry points and link them together using `spirv-link`.
    - We could use a different shader language, such as by using HLSL, slang, Circle-C++, or rust-gpu.
    
That's all the GLSL modification we'll need for this tutorial!

### Set the Instance Shader Binding Table Offset

As mentioned above, `instanceShaderBindingTableRecordOffset` is added to the index of the record used in the hit group SBT. Currently, each instance uses a `instanceShaderBindingTableRecordOffset` of 0, meaning that they always use the hit record with index 0. To have each instance use a random material, set the `instanceShaderBindingTableRecordOffset` as follows:

~~~ cpp
      // An offset that will be added when looking up the instance's shader in the SBT.
      instance.instanceShaderBindingTableRecordOffset = uniformIntDist(randomEngine);
~~~

### Loading More Shader Modules

Set `NUM_C_HIT_SHADERS`, the number of closest hit shaders, to 9:

~~~ cpp
  const size_t                                      NUM_C_HIT_SHADERS = 9;
~~~

Then load each of the shader modules and set their debug names as follows:

~~~ cpp
  std::array<VkShaderModule, 2 + NUM_C_HIT_SHADERS> modules;
  modules[0] = nvvk::createShaderModule(context, nvh::loadFile("shaders/raytrace.rgen.glsl.spv", true, searchPaths));
  debugUtil.setObjectName(modules[0], "Ray generation module (raytrace.rgen.glsl.spv)");
  modules[1] = nvvk::createShaderModule(context, nvh::loadFile("shaders/raytrace.rmiss.glsl.spv", true, searchPaths));
  debugUtil.setObjectName(modules[1], "Miss module (raytrace.rmiss.glsl.spv)");
~~~ cpp highlight
  for(int closestHitShaderIdx = 0; closestHitShaderIdx < NUM_C_HIT_SHADERS; closestHitShaderIdx++)
  {
    const int         moduleIdx = 2 + closestHitShaderIdx;
    const std::string filename  = "shaders/material" + std::to_string(closestHitShaderIdx) + ".rchit.glsl.spv";
    modules[moduleIdx]          = nvvk::createShaderModule(context, nvh::loadFile(filename, true, searchPaths));

    const std::string debugName = "Material " + std::to_string(closestHitShaderIdx) + " shader module";
    debugUtil.setObjectName(modules[moduleIdx], debugName);
  }
~~~

### Adding Shaders to the SBT

Then add these shaders to the list of shader stages:

~~~ cpp
    // Stage 1 will be the miss shader.
    stages[1]        = stages[0];
    stages[1].stage  = VK_SHADER_STAGE_MISS_BIT_KHR;  // Kind of shader
    stages[1].module = modules[1];                    // Contains the shader
~~~ cpp highlight
    // Stages 2 through the end will be closest-hit shaders.
    for(int closestHitShaderIdx = 0; closestHitShaderIdx < NUM_C_HIT_SHADERS; closestHitShaderIdx++)
    {
      const int moduleIdx      = 2 + closestHitShaderIdx;
      stages[moduleIdx]        = stages[0];
      stages[moduleIdx].stage  = VK_SHADER_STAGE_CLOSEST_HIT_BIT_KHR;
      stages[moduleIdx].module = modules[moduleIdx];
    }
~~~

Finally, add each new shader group to the array of groups:

~~~ cpp
    // CLOSEST-HIT REGION
~~~ cpp highlight
    // Group N - uses Stage N as its closest-hit shader
    for(int closestHitShaderIdx = 0; closestHitShaderIdx < NUM_C_HIT_SHADERS; closestHitShaderIdx++)
    {
      const int moduleIdx                = 2 + closestHitShaderIdx;
      groups[moduleIdx]                  = nvvk::make<VkRayTracingShaderGroupCreateInfoKHR>();
      groups[moduleIdx].type             = VK_RAY_TRACING_SHADER_GROUP_TYPE_TRIANGLES_HIT_GROUP_KHR;
      groups[moduleIdx].closestHitShader = moduleIdx;  // Index of closest-hit in `stages`
    }
~~~

This concludes the code! The ray tracing pipeline and SBT will now be created with the new closest hit shaders, and `traceRayEXT` will now automatically call the new closest hit shaders.

If all went well, you should now be able to build and run the application and see the following:

![441 instances with 9 different materials rendered using ray tracing pipelines.](images/e11-output-3.png)

Measuring performance using Nsight Systems (see Section [Measuring Performance] for an Nsight Systems tutorial) on an RTX 3090, I measured that this chapter's code rendered this scene in 761ms, while Chapter [Multiple Materials]'s ray query-based code took 811ms, using the code from the `e11_rt_pipeline_3` and `e10_materials` folders. In other words, without changing the structure of the path tracer, using ray tracing pipelines resulted in a 6% performance increase!

## Conclusion

This concludes the extra `vk_mini_path_tracer` chapters! I hope you've enjoyed this deeper look into ray tracing and Vulkan.

If you're interested in some of the aspects of Vulkan ray tracing we didn't cover here, like animation and interaction, any hit shaders, callable shaders, and glTF, check out the Extra Tutorials of the [NVIDIA Vulkan Ray Tracing Tutorials](https://github.com/nvpro-samples/vk_raytracing_tutorial_KHR) by Martin-Karl Lefrançois, Pascal Gautron, Neil Bickford, and David Akeley.

There are many more resources for Vulkan and ray tracing online. The Khronos Group maintains a list of Vulkan educational resources [here](https://www.khronos.org/vulkan/educational-resources). On the topic of path tracing itself, Peter Shirley has written three introduction-to-ray-tracing books, all of which are free online [here](https://raytracing.github.io/). If you've finished this tutorial, [Chapter 10 of Ray Tracing in One Weekend](https://raytracing.github.io/books/RayTracingInOneWeekend.html#dielectrics) is a good place to continue. These books implement a CPU-based renderer, but a Vulkan implementation is available [here](https://github.com/GPSnoopy/RayTracingInVulkan), and the underlying techniques don't depend as much on the API. (In addition to speed, using Vulkan ray tracing also means that you can skip chapter 3 of *Ray Tracing: The Next Week*.)

If you're interested in finding out how to implement other techniques in Vulkan, the [NVIDIA DesignWorks Samples page](https://github.com/nvpro-samples/build_all) (of which this is a part) includes many other Vulkan and OpenGL samples, from [using the OptiX 7 AI denoiser](https://github.com/nvpro-samples/vk_denoise) to [device-generated commands](https://github.com/nvpro-samples/vk_device_generated_cmds) to [OpenGL-Vulkan interop](https://github.com/nvpro-samples/gl_vk_raytrace_interop).

</script>

<!-- Markdeep: -->
<link rel="stylesheet" href="vkrt_tutorial.css?">
<script> window.markdeepOptions = { tocStyle: "medium" };</script>
<script src="markdeep.min.js" charset="utf-8"></script>
<script src="https://developer.download.nvidia.com/ProGraphics/nvpro-samples/scripts/markdeep.min.js" charset="utf-8"></script>
<script>
    window.alreadyProcessedMarkdeep || (document.body.style.visibility = "visible")
</script>